A scheduled job is server-side work the instance runs on a timetable: a nightly export, an hourly clean-up, a report emailed every Monday. ServiceNow stores job definitions in the Schedule Job (sysauto) table and its children, with Scheduled Script Execution (sysauto_script) being the one developers write most. When a job is due, the scheduler on one of the application nodes picks up its schedule item from sys_trigger and runs it on a worker thread.
That’s simple to set up, which is why many instances have jobs that started as five-line scripts and grew into hour-long monsters nobody dares touch. Interview questions here are about what happens when things go wrong: the job runs for too long, the node restarts halfway, the external system gets the same record twice, and nobody can tell what the last run actually did.
How a job runs
Each scheduled script has a schedule (daily, weekly, periodically every N minutes, on demand and so on), a “Run as” user, an optional condition script, and the script itself. ServiceNow’s documentation lists the states a job moves through: Ready, Queued, Running and Error. When the run finishes, the scheduler calculates the next run time.
Three properties of that model drive the design advice:
- Workers are shared. Each node has a limited pool of scheduler worker threads, used by every job on the instance, including platform jobs such as event processing and email. A job that runs for an hour holds a worker for an hour.
- The script runs in one transaction context. If it tries to process everything in a single run, all of its work is at risk when anything interrupts it.
- The “Run as” user matters. GlideRecord in a job doesn’t check ACLs, but the user still appears in
sys_updated_by, in audit history, and in business rules that test the current user. A dedicated system account makes those records honest.
Bounded batches, not one long run
Suppose a job sends contract renewal records to a billing API every night: 12,000 records at about 40 ms per call. Done naively, it loops over everything in one run, and halfway through the night a node restarts during an upgrade or a patch. I modelled that in Node, with the restart hitting after 7,300 calls. The first design processes everything in one run and has no record of progress, so the retry starts again from the top. The second processes 500 records per run and saves its progress only after each batch:
while (watermark < N) {
const slice = records.filter((r) => r.id > watermark).slice(0, batch);
// ...send each record; the simulated restart interrupts call 7,300...
if (!crashed) watermark = slice.at(-1).id; // committed only after the whole batch succeeded
}unbounded {"calls":19300,"longestRunSec":480,"duplicates":7300}
batch of 500 {"calls":12300,"runs":25,"longestRunSec":20,"duplicates":300}The single long run held a worker for eight minutes at a time and, after the restart, sent 7,300 records to billing twice. The batched job never ran longer than 20 seconds, needed 25 runs to finish, and repeated only the 300 records from the batch that was interrupted.
Three hundred duplicate invoices is still a bad morning, so the next step is to record progress per record rather than per batch. Then a restart repeats at most the one record that was in flight. And because even that can happen, the billing call itself should carry an idempotency key (the contract number plus renewal period, say) so the far side ignores a repeat. Batching limits the damage, per-record status shrinks it, and idempotency removes it.
A job script that follows those rules
Here’s the shape on the instance. It’s illustrative: no instance was available while writing it, and u_billing_status and the BillingClient script include stand in for your own names.
// Scheduled Script Execution: "Send contract renewals to billing" (every 10 minutes)
(function () {
var BATCH = 500;
var started = new GlideDateTime();
var sent = 0, failed = 0;
var gr = new GlideRecord('u_contract_renewal');
gr.addQuery('u_billing_status', 'pending');
gr.orderBy('sys_created_on');
gr.setLimit(BATCH);
gr.query();
var client = new BillingClient();
while (gr.next()) {
try {
client.sendRenewal(gr, gr.getValue('u_number') + ':' + gr.getValue('u_period')); // idempotency key
gr.setValue('u_billing_status', 'sent');
sent++;
} catch (e) {
gr.setValue('u_billing_status', 'error');
gr.setValue('u_billing_error', String(e).substring(0, 1000));
failed++;
}
gr.update();
}
var run = new GlideRecord('u_job_run');
run.initialize();
run.setValue('u_job', 'billing_renewals');
run.setValue('u_started', started);
run.setValue('u_sent', sent);
run.setValue('u_failed', failed);
run.insert();
gs.info('[billing_renewals] sent={0} failed={1}', sent, failed);
})();Each run handles at most 500 records, picks up where the last one stopped because the query only finds pending rows, and marks each row as soon as its call succeeds. A row that fails moves to error instead of being retried forever inside the same run, so one broken record can’t block the queue. The run summary goes into a small table, which means you can build a report on it and alert when u_failed is non-zero or when no run has been recorded for an hour. A line in the system log is useful too, but logs are hard to query and are rotated.
Interview tip
Put the batch size and any on/off switch in a custom settings table rather than a system property if people will change them often, because every property change flushes caches across the cluster.
Overlap and long runs
The scheduler normally won’t start the same schedule item again while it’s still running; the next run is calculated after the current one finishes. Overlap still happens in practice, though. Someone clicks “Execute Now” while the scheduled run is active. Two jobs were created for the same work, one copied from the other. A business rule triggers the same script include the job uses. If the work can overlap, design for it: claim records by setting a status such as in_progress with the run’s ID before working on them, and have the query skip claimed rows. That makes double processing a non-event rather than a race.
Watch for jobs whose run time creeps up. A job that took 30 seconds in January and four minutes in October is telling you the data grew and the batch size or query didn’t adapt. Typical causes are a missing index on the filter field, a query without setLimit, or nested GlideRecord queries inside the loop. Adding a duration column to your run table makes that trend visible.
When something fails at 2 a.m.
The questions you’ll be asked the next morning are: did it run, what did it do, and what’s left? A job built like the one above answers all three. The run table shows each run with counts. Records in error hold the reason. Records still pending are the remaining work, and the next run will take them. Fixing the cause and setting failed rows back to pending is a safe retry, because the billing side deduplicates.
Compare that with the common alternative: a job that logs “Started” and “Finished” and nothing in between, processes everything each time, and keeps no per-record status. When it fails, nobody knows which records were sent.
Interview practice: the job that doubled invoices
A nightly job sends 12,000 records to an external billing system. Last night an instance upgrade restarted nodes while it was running, and finance reports that roughly 7,000 customers were invoiced twice. How do you fix the job so this can’t happen again?
Answer
The duplicates mean the job had no record of what it had already sent: after the restart it started from the beginning and resent everything processed before the interruption. I’d fix it in layers. First, give each record a status (pending, sent, error) and update it right after each successful call, so a restart only risks the record in flight. Second, process a bounded batch per run, say 500 every ten minutes, so no single run holds a scheduler worker for long and each run’s effect is small. Third, send an idempotency key with each call, built from the contract number and billing period, and ask the billing team to ignore repeats; that covers the one in-flight record and any manual “Execute Now” overlap. Finally, write a run summary with counts to a table and alert on failures or missed runs. For last night’s damage, I’d give finance the list of records sent twice, which the new status and run history would have made obvious.