Languages and dialects

spec/languages.toml is the canonical table. The table below is generated from that file, while the published pre-commit hooks pass every text file to the CLI detector so reserved names and extensionless shebang scripts are covered too. The binary embeds that same file, so ocomment languages prints these rows in columns and ocomment languages --format json prints them as JSON.

A dialect changes the lexical rules rather than the file type: --dialect mysql is still SQL, and only that dialect treats /*!40101 ... */ as something the server executes rather than as a comment.

A language is chosen from the file extension, and --language overrides that for a run — which is what ocomment strip needs, because standard input has no name. --dialect picks the dialect for the same run, and [languages.<name>] dialect = "..." in .ocomment.toml picks one for everybody working in the repository. An incompatible pair is an error rather than a silent fallback:

$ ocomment strip --language rust --dialect mysql
ocomment: unsupported dialect `mysql` for rust; supported: standard

OComment has 30 built-in languages covering 79 file extensions and 18 named dialects.

Detected without an extension

A file whose extension decides nothing is looked up by its whole name, and a file with no name at all — a script on standard input — is read from its #! line. A name is matched without regard to case. A direct shebang uses only the executable basename; for /usr/bin/env, options, assignments, --, and -S/--split-string are consumed to find the executable it actually launches. Parent directories, option values, and program arguments are never searched for an interpreter-looking name.

Anything else

HTML is scanned recursively: the contents of a <script> element are scanned as JavaScript and the contents of a <style> element as CSS, each with the comment forms of that language rather than of HTML.

PHP is scanned in the code half of the file only. What sits between <?php (or <?=) and ?> is scanned for PHP comments; the inline HTML around those tags is content, so an HTML <!-- ... --> comment in a PHP file is not reported and is never removed. Which mode a byte sits in is decided by everything above it, so only a line break in inline HTML is a point an editor may rescan a PHP file from: a file that is all PHP is rescanned from the top.

Ruby is scanned with a lexer that keeps four states, because four of Ruby’s tokens are spelled with a byte that is also an operator and only where the token stands decides which: / is a regular expression or a division, % a literal or a modulo, ? a one-character string or a ternary, and << a here document or an append. The fourth state is the one alias and undef leave Ruby in, where %s opens a symbol literal however it is spaced while %w, %q and / after the same keyword stay operators. Ruby’s own parser answers those from a lexer state that a symbol table feeds – it knows whether a is a local variable or a method – and a scanner has no symbol table, so a bare word is always read as a method that may take a command argument. a /b/ c is therefore a pattern here where Ruby, knowing a to be a variable, reads two divisions. The reading that differs is the one that keeps more bytes inside a literal, so a comment is never invented out of a division; it is a comment left unfound, not a byte removed.

Three smaller readings go the same way. A / inside a character class stays inside the pattern, where Ruby’s own lexer ends the literal at it, so /[/]/ is one regular expression. $#, which Ruby refuses outright as a global variable name, is read as a $ and then the comment that # opens everywhere else. And a here document terminator may be spelled with digits, because Ruby builds an unquoted one out of name bytes and a digit is one of those from the first byte on: puts <<2 opens a here document that runs to a line reading 2, and every line between the two is body rather than code. Where the same << follows an operand – a[0] <<2, p 1 <<2 – it is the shift operator it looks like, and the rest of that line is code.

Zig is the one built-in language with no block comment. /* is the division operator followed by multiplication, so a /* ... */ written in a Zig file is code: it is not reported and it is never removed. /// and //! are documentation comments and a fourth slash takes the marker back, so //// is an ordinary one. A multiline string literal is read one line at a time: a \\ wherever a token may begin runs to the end of that line as string content, and the line under it starts in code again. zig fmt: off and zig fmt: on are kept, and they are matched as the whole phrase the formatter compares rather than as a prefix of it.

R has one comment token and # opens it, so what the scanner has to know is the four literals that carry a # as content. Two are quoted strings and both may run over a line break; the third is a backquoted name, which is lexed the same way; and the fourth is the %...% operator, whose name is every byte up to the next % on the same line – x %a # b% y is one operator, and a % with no second % before the line break is an error rather than a comment opener. A raw string is r or R, a quote, any run of dashes and one of (, [ or {, and it closes only on the matching bracket with the same run of dashes and the same quote. The r opens one only where it begins a token, so the quote in xr"(a)" opens an ordinary string instead. #' is roxygen2’s documentation marker and is a doc-line; R’s own parser calls every one of these a comment and draws no distinction.

Dart is the one built-in C-family language whose block comment nests, so /* /* */ */ is one comment and commenting out a region that already holds one works. Its documentation markers are /// and /**, decided at the single byte behind the opener: a fourth slash leaves //// documentation, where Lua’s ---- and Zig’s //// take the marker back, and //! and /*! document nothing. A string is written six ways – either quote, single-line or triple-quoted, raw or not – and only the r of a raw one takes away the \ escape and ${ ... } interpolation. That interpolation is code, so a comment written inside one is a comment, and a // there ends at the line break while the string around it carries on below. # opens a symbol literal and is a comment only as the #! script tag at the very first byte of a file. // @dart = 2.12 is kept because the language version it names changes what the rest of the file means, and // dart format off and // dart format on are matched as the whole phrase dart_style compares rather than as a prefix.

Swift’s block comment nests as Dart’s does, /// and /** document – //// still does and the empty /**/ does not, since its second * is the first byte of its own terminator – and //! and /*! document nothing. A string is written four ways: single line or """, each of them raw or not, where raw is a run of # in front of the quote. Those hashes do not take the escape and the interpolation away, they rename them: with one hash the escape is \# and \#( opens the interpolation, the closing delimiter needs the same run behind the quote, and a bare \( is content. The interpolation is code, so a comment written inside one is a comment.

The regular expression literal is the one thing in Swift that can carry a // with no quote in front of it, and only JavaScript’s scanner faces the construct at all besides this one. #/ ... /# is the extended form, which may hold an unescaped / and, when its opener ends the line, may span lines. The bare / ... / ends at the first unescaped /, never crosses a line, and may not begin with a space or a tab (The Swift Programming Language, Lexical Structure) – and its last two bytes may still spell //, because /a\// is a literal whose content is a\/. One rule cannot be had from the bytes alone: whether a / in an ambiguous position is a literal at all is settled in Swift by the parser, and this is a lexer. It reads a / as a literal exactly where a prefix operator may stand and where the content closes on the same line; every case that decides differently from the compiler is a file swiftc rejects, where the compiler is lexing a literal it has already diagnosed and this reads the comment inside it instead – so a fix on a file that does not compile can take the two bytes of a comment opener a repaired file would have kept. The corpus case swift-bare-regex-limitation records one. ' is no delimiter of the language, but the compiler lexes '...' anyway so that it can offer a fix-it, and this follows it for the same reason.

C# is the one built-in language whose lines are lexed two ways. A line whose first non-blank byte is # is a pre-processing directive, and ECMA-334 6.5.1 ends one with PP_Whitespace? SINGLE_LINE_COMMENT? New_Line: a // is the only comment it can carry, a /* on it opens nothing, and a " opens a string that takes no \ escape and ends at the line – which is what keeps the // inside #line 1 "a//b.cs" out of reach. Four directives take the rest of their line as a message instead: #error and #warning carry the text a diagnostic quotes, and #region and #endregion the label an editor folds under, so #region // x carries a comment and #region x // y does not. A conditional section is scanned as ordinary code rather than skipped, for the reason #if 0 is in C and C++: which symbols a build defines is not in the file, and code is what an #if DEBUG body is in every build that defines the symbol. The price is a section written to be skipped rather than compiled – prose, or another language – whose bytes are not C#: an apostrophe in one opens a character literal its line does not close, so the file is called invalid and no edit is offered for it, where Roslyn reports the whole section as one blob of disabled text and finds nothing in it. Refusing to edit is the safe half of being wrong, and two files of the 70,630 measured are affected – both of them a block of prose under an #if false, and the only two this scanner calls invalid that Roslyn does not. The corpus case csharp-conditional-section-limitation records one.

A string is written eight ways – plain, verbatim, raw, and each of those interpolated – and the three rules differ in what closes them. A plain one takes the \ escape, which carries the character behind it in whatever it is, a line terminator included; a verbatim @"..." spells its quote "", takes no escape, and carries line breaks; and a raw one is opaque until a run of at least as many quotes as its opener carried comes back, carrying line breaks only when its opener ends a line. Interpolation is a switch on top of all three: a run of n $ in front of the quote makes a run of n braces the thing that opens a hole, so { is content in a $$""" literal and {{ is code. A hole is code, so a comment written in one is a comment and may carry a line break the text around it could not – but the format clause behind the first : of a hole is text again, which is why the // in $"{x:D4 // n}" is not a comment and why $"{global::X}" needs the parentheses the compiler asks for.

C# also counts five line terminators where every other C-family scanner here counts two: ECMA-334 6.3.1 adds U+0085, U+2028 and U+2029 to the carriage return and line feed, and Roslyn ends a // comment at all five. A scanner that read on past one would swallow the code behind it on the same physical line, and a removal would take that code with it.

Scala’s block comment nests as Dart’s does, and its documentation comment is the one the Scala 3 compiler’s comment reader answers to: a comment is documentation exactly when its text starts with /** (Comment.isDocComment), so /**/ and /***/ are documentation comments and /// – which scaladoc does not read – is an ordinary line comment.

A Scala string is interpolated exactly when an identifier stands directly before its quote: the compiler’s lexer turns that identifier into INTERPOLATIONID, so s"...", raw"..." and a custom interpolator such as xml"..." interpolate, while a keyword – its own token – and a number leave the quote to a plain string whose $ is content. Inside an interpolated string $$ and $" write a literal $ and ", ${ ... } opens an expression that is code – a comment written there is a comment – and $ followed by an identifier starts another one. A triple-quoted string closes on the first three quotes of a run and makes any further quotes of the run part of its value, so """a"""" is the string a"; a backquoted identifier may hold // without it being a comment.

The XML literal is the one Scala construct whose text is not code: the compiler’s lexer emits an XMLSTART token and the parser re-reads the literal with an XML scanner, so this scanner reads it the way the parser does. Element text, CDATA and processing instructions are opaque, { ... } in text or an attribute is code, <!-- ... --> is an XML comment, and the literal ends at the close tag matching its root or at a self-closing />. A literal begins exactly where the lexer says one does: a < preceded by space, tab, line feed, {, ( or > and followed by an XML name start, ! or ?.

Vue and Svelte components are HTML with code in the template: a <!-- ... --> is an HTML comment, and {{ ... }} in Vue or { ... } in Svelte opens an expression whose comments are comments — in Vue, a v-pre element makes its whole content raw text instead. The <script> and <style> bodies are scanned as their own languages, the lang attribute choosing which: ts and tsx select TypeScript, jsx JavaScript with JSX, and scss and the indented sass the SCSS dialect. A lang this scanner has no rules for — coffee, less, pug — makes the whole block opaque.

SCSS and the indented Sass syntax are CSS plus two rules: // opens a silent comment, and #{ ... } opens an interpolation whose expression is code. An unquoted url( ... ) is read the way dart-sass reads it — its bytes are URL text until the ) that ends them, a protocol-relative url(//cdn/x.png) included — with #{ ... } inside it code.

Markdown is scanned per CommonMark: an HTML comment is a comment, a fenced code block is scanned as the language its info string names — ```rust, {r} and c++ all reach their scanners — and an inline code span or an indented code block is opaque, so a // or a /* inside one is code text, not a comment.

Perl is scanned conservatively: a # runs to the end of its line, a POD block is opaque, and every quote word — the single and double quotes and backticks, q, qq, qw, qx, m, s, tr and y with delimiters of their own, and the here-documents — hides a # written inside it. A / directly after a closing parenthesis, bracket or brace is reported as lexically ambiguous: perl reads f() /a#b/ as a regular expression and (2) / 2 as a division, and only the parse context tells which, so the file is called invalid and nothing is edited.

YAML is scanned lexically, and valid is a lexical answer: the shapes a YAML parser rejects are not all shapes a lexer can see. A comment line inside a multi-line plain scalar makes the file a parse error while it is there, and taking it away leaves a scalar that parses and folds the two halves into one value; that comment is reported and removed like any other.

The block scalar is the other way round, and it is the one place in any language where the hole a removal leaves carries meaning. A block scalar decides where its body ends from the lines below it (YAML 1.2.2, 8.1.1), so a whole-line comment under a body is what terminates it – and whatever a removal writes on that line is read back as part of the value. A line of spaces as wide as the comment, which columns writes, is indented at least as deep as the body whenever the comment was wide enough. An empty line, which lines writes, is content under |+ and >+, which keep every empty line trailing a body (8.1.1.2).

So the rule is one rule, and it holds whatever the block scalar chomps: a whole-line comment sitting in the run of blank and comment lines under a block scalar body is removed by taking its whole line, terminator and all, under every layout. Those lines are the one place lines does not keep a line’s number and columns does not keep a column’s – both give that up rather than give up the value. Under |+ and >+ the removal also takes the blank lines the comment was sheltering, which become content the moment it is gone; the blank lines above the first comment were content already and are left exactly where they were.

There is one comment that rule cannot reach, and it is kept instead. The line a body ends at is a comment shallower than the body’s own content; take it away and the lines under it are read against the body again. When one of those is a comment the run keeps, and it is indented to the content depth, the body swallows it and the value grows a line. No removal preserves the value there, so the comment that ends the body is kept, with the reason structural in a YAML block scalar trail – the one keep --policy all does not overrule.

The depth this is measured at is the body’s content indentation: the explicit indentation indicator when the header spells one out, and otherwise the indentation of the body’s first non-empty line (YAML 1.2.2, 8.1.1.1). A surviving comment shallower than that is outside the scalar before and after the removal, so nothing above it is kept.

tools/yaml_roundtrip.py is what holds this to its word: it strips thousands of generated YAML documents under every layout and every policy and asserts that PyYAML reads the same value out of each one before and after.

A delimiter-based syntax that is not in the table above can be described declaratively as a profile in .ocomment.toml, which needs no code; see Configuration. A syntax whose comments cannot be described by delimiters alone needs a scanner plugin instead; see Plugins.