JavaScript RegExp Stateful Flags and lastIndex

Here’s a bug report that looks impossible. A form validates product codes with a regex, and the same valid code passes, then fails, then passes again on successive clicks. Nothing about the input changes. The regex is a constant. The only odd thing is a g at the end of the pattern that somebody added because “g means it finds all the matches”.

The cause is a property most developers never look at: lastIndex. Regex objects with the g (global) or y (sticky) flag remember where the last match ended, and test and exec start their next search from there. This guide shows exactly how that state works, where it’s useful, and how to avoid it when it isn’t.

The alternating test() bug

Here’s the product-code check, trimmed to the essentials. Codes look like AB-1234:

const isSku = /^[A-Z]{2}-\d{4}$/g;

console.log(['AB-1234', 'AB-1234', 'AB-1234'].map(s => isSku.test(s)));
// [ true, false, true ]

Three identical valid inputs, and the middle one fails. Trace lastIndex through each call:

1. lastIndex starts at 0. The search starts at position 0, the pattern matches the whole 7-character string, and lastIndex is set to 7 (the index just after the match). Returns true.
2. The search starts at position 7. The string is only 7 characters long, so there’s nothing left to match. On failure, test resets lastIndex to 0 and returns false.
3. The search starts at 0 again and succeeds. Returns true.

The state lives on the regex object, not on the string, so it carries across different inputs too:

console.log(['AB-1234', 'CD-5678', 'EF-9012'].filter(s => isSku.test(s)));
// [ 'AB-1234', 'EF-9012' ]

CD-5678 is a valid code and got filtered out, purely because it came straight after a successful match. In the real bug report, the regex was a module-level constant shared by every call, so whether a code passed depended on what the previous user typed.

The fix is to remove the g. A validation regex anchored with ^ and $ has at most one possible match, so “global” has no meaning for it. Without g or y, test and exec ignore lastIndex entirely and always start at 0:

const once = /cat/;
console.log(once.test('cat'), once.test('cat'), once.lastIndex);  // true true 0

You’ll also see people fix it with isSku.lastIndex = 0 before each call. It works, but it treats the symptom; someone will copy the regex without the reset.

Why a regex literal inside a function doesn’t have this bug

A regex literal is evaluated each time its expression runs, creating a new object. So function check(s) { return /x/g.test(s); } returns true on every call, because each call gets a fresh regex with lastIndex 0. The bug appears when the regex is created once and reused: a module constant, a class field, or a pattern passed in as an argument. That’s also why it often shows up after a refactor that “hoisted the regex for performance”.

Iterating over matches with exec and lastIndex

The stateful behaviour exists for a reason: it’s how exec walks through every match in a string. Each call finds the next match after lastIndex, returns its details, and moves lastIndex forward:

const re = /(\w+)=(\d+)/g;
const q = 'a=1&b=22&c=333';
let m;
while ((m = re.exec(q)) !== null) {
  console.log(m[1], m[2], 'next search starts at', re.lastIndex);
}
// a 1 next search starts at 3
// b 22 next search starts at 8
// c 333 next search starts at 14

When no match remains, exec returns null and resets lastIndex to 0, so the loop ends and the regex is ready for reuse. If you break out of the loop early, though, lastIndex stays wherever it was, and the next caller of that regex starts mid-string.

This loop has a known trap with patterns that can match the empty string. With /a*/g, an empty match leaves lastIndex where it was, so exec finds the same empty match forever and the loop never ends. Hand-written loops have to detect m[0] === '' and bump lastIndex by one.

matchAll is the modern replacement

String.prototype.matchAll does the same iteration without any of the shared-state hazards:

console.log([...q.matchAll(/(\w+)=(\d+)/g)].map(m => `${m[1]}:${m[2]}@${m.index}`));
// [ 'a:1@0', 'b:22@4', 'c:333@9' ]

It clones the regex internally, so the original’s lastIndex is never read or changed. It handles empty matches correctly: [...'baa'.matchAll(/a*/g)] yields three matches (an empty one at 0, 'aa' at 1, an empty one at 3) and stops. And each result includes capture groups and the index, which match with g doesn’t give you.

matchAll requires the g flag and throws if it’s missing:

q.matchAll(/(\w+)=(\d+)/);
// TypeError: String.prototype.matchAll called with a non-global RegExp argument

replaceAll has the same requirement and the same error message, apart from the method name. That’s deliberate: a non-global regex in a method called “all” is almost certainly a mistake, so the language refuses rather than silently replacing one match.

How the string methods treat the flag

The string methods differ in whether they look at lastIndex. match without g returns the first match with its groups (['a=1', 'a', '1']); with g it returns all matched strings and no groups (['a=1', 'b=22']). replace with a g regex replaces every match and resets lastIndex to 0 afterwards. search ignores both g and lastIndex and always reports the first match position. Only test and exec called directly on the regex depend on the leftover state, which is why the bug almost always involves one of those two.

The sticky flag for tokenizers

The y flag also uses lastIndex, but more strictly: the match must begin exactly at lastIndex, with no scanning ahead.

const sticky = /\d+/y;
sticky.lastIndex = 2;
console.log(sticky.exec('ab12cd')?.[0], sticky.lastIndex);  // 12 4
sticky.lastIndex = 1;
console.log(sticky.exec('ab12cd'));                         // null

At position 2 there are digits, so it matches. At position 1 there’s a b, and a sticky regex doesn’t go looking further along the string. A g regex would have skipped ahead and found 12 anyway.

That strictness is exactly what a lexer needs. You want to know what token starts at the current position, not whether one appears somewhere later. Here’s a small tokenizer for arithmetic expressions:

function tokenize(src) {
  const rules = [['num', /\d+/y], ['op', /[+*()-]/y], ['ws', /\s+/y]];
  const out = [];
  let pos = 0;
  outer: while (pos < src.length) {
    for (const [type, re] of rules) {
      re.lastIndex = pos;
      const m = re.exec(src);
      if (m) {
        if (type !== 'ws') out.push([type, m[0]]);
        pos = re.lastIndex;
        continue outer;
      }
    }
    throw new SyntaxError(`Unexpected ${JSON.stringify(src[pos])} at ${pos}`);
  }
  return out;
}

tokenize('12 + 3*(4-1)');
// [['num','12'], ['op','+'], ['num','3'], ['op','*'], ['op','('], ['num','4'], ['op','-'], ['num','1'], ['op',')']]
tokenize('2 ^ 3');
// SyntaxError: Unexpected "^" at 2

Each rule is tried at the current position by setting lastIndex explicitly. Here the state on the regex is a feature, and setting it before every exec means no leftover value can leak in. Without y, the digit rule could skip over the ^ and match 3, and the tokenizer would silently accept invalid input.

Practice question: the intermittent password check

Question. A sign-up form shares a list of rules between the browser and a Node API: export const rules = [/[A-Z]/g, /\d/g, /[^\w\s]/g], checked with rules.every(r => r.test(password)). QA reports that after an API restart the first sign-up works, and then most valid passwords are rejected by the API even though the browser accepted them. What’s happening, and what’s the fix?

Answer. The g flag makes each regex remember lastIndex between calls, and on the server the module is loaded once and shared across every request. After a password matches /[A-Z]/g at, say, index 0, that regex’s lastIndex is 1, so the next request’s check starts from position 1 and fails if that user’s only capital letter is the first character. A failed test resets that one regex, but every stops at the first failure, so the regexes after it keep stale lastIndex values from earlier requests and fail the next time instead. Running 'Abc1!' through the check four times in a row in Node gives [true, false, false, false]. The browser seemed fine because each page load gets a fresh module and a user usually submits once. The fix is to drop g from all three; these are yes-or-no checks that need no iteration. A good answer adds a regression test that calls the validator twice in a row with the same valid password, because that’s the smallest case that reproduces the bug.

Further reading

• JavaScript interview questions
• JavaScript MCQs
• The iterator protocol
• Module live bindings
• MDN reference: RegExp: lastIndex