The language server

How larva lsp serves editors - the JSON-RPC transport, the methods it answers, projects and analysis per change, language pack requests, decompiled larva documents, semantic tokens and the borrow check as you type.

Last updated 16 min read Edit on GitHub

On this page

larva lsp is a Language Server Protocol server over standard input and output. One server serves every language installed on Larva. It is split in two layers:

  • larva (Larva.Cli/Lsp, class LspServer) keeps the protocol, the open documents, the .queen projects, the decompiled files and the borrow checker. It knows no language's syntax.
  • The language pack answers everything that needs the syntax: analysis, completion, hover, definitions, signatures, the outline, semantic tokens and decompiling. It does that with the shared editor services of Larva.Hir.Services (Editor, Catalog, Views), so features that only need the HIR catalog work the same way in every language.

The reason for the split comes from the design: before Larva was a separate toolchain, the server was built on one language's binder. Moving project handling, references and navigation into larva means a new language gets a working editor by answering a few lsp.* requests, and .queen files, bindings and decompiled libraries behave the same for all of them.

 editor (VS Code: the Larva extension)
    │  JSON-RPC over stdio: Content-Length framing
    ▼
 LspServer  (larva lsp)
    ├─ documents         uri → text (full sync)
    ├─ projects          .queen → Project (sources, references, options), cached by stamp
    ├─ QueenFiles        .queen completion / hover / QN diagnostics
    ├─ decompiled files  %LOCALAPPDATA%\Larva\decompiled\<assembly>\<Type>.<ext>, served as larva: URIs
    ├─ borrow checker    background compile + lower + BorrowChecker.Check
    │
    │  pack.Call("lsp.analyze" | "lsp.completion" | … , JSON)
    ▼
 language pack (Languages/<Name>/: Antril.dll …)
    ├─ workspace per key  (catalog of System.Core, System.Bindings, assembly: libraries)
    ├─ IDocumentAnalysis  per open document
    └─ LanguageService    completion lists, hovers, definitions, signatures (Larva.Hir.Services)

Starting it

Terminal
larva lsp [--core <folder>]

--core points at a System.Core other than the installed one. Everything written to the console by mistake goes to standard error: LspServer.Run takes the real standard output for the protocol and redirects Console.Out to Console.Error, so a stray message from a library cannot corrupt the stream.

The VS Code extension larva.larva starts it as larva lsp with the stdio transport. Its document selector holds .queen files and, for each language an installed extension declares under "larva": { "languages": [...] }, the file, untitled and larva schemes of that language. The Antril extension declares antril; it has no server code of its own.

Transport

Messages are framed as the protocol says: header lines up to an empty line, Content-Length: n, then n bytes of UTF-8 JSON. Read parses the headers byte by byte, reads the body, and parses it with JsonNode.Parse. Send writes header and body under a lock, because borrow-check results are published from a background task.

The messages are handled as generic JSON trees, never mapped to typed records. The installed larva is compiled ahead of time without run-time type metadata for serialization, so mapping a typed record to JSON could fail there even when it works in a working copy; building the trees by hand keeps both builds identical.

An exception while handling a message is printed to standard error; when the message was a request, the server answers with error -32603 and the exception's message. An unknown request gets -32601:

json
{"jsonrpc": "2.0", "id": 5, "error": {"code": -32601, "message": "unknown method textDocument/foo"}}

Initialize

The initialize answer from larva 0.8.0 with the Antril pack installed:

json
{
 "jsonrpc": "2.0",
 "id": 1,
 "result": {
  "capabilities": {
   "textDocumentSync": {
    "openClose": true,
    "change": 1,
    "save": {
     "includeText": false
    }
   },
   "completionProvider": {
    "triggerCharacters": [
     ".",
     ":"
    ]
   },
   "hoverProvider": true,
   "definitionProvider": true,
   "signatureHelpProvider": {
    "triggerCharacters": [
     "(",
     ","
    ]
   },
   "documentSymbolProvider": true,
   "workspaceSymbolProvider": true,
   "semanticTokensProvider": {
    "legend": {
     "tokenTypes": [
      "namespace",
      "type",
      "class",
      "enum",
      "interface",
      "struct",
      "typeParameter",
      "parameter",
      "variable",
      "property",
      "enumMember",
      "function",
      "method",
      "field",
      "keyword",
      "event",
      "exception",
      "attribute",
      "bitflags"
     ],
     "tokenModifiers": [
      "declaration",
      "static",
      "readonly",
      "defaultLibrary"
     ]
    },
    "full": true
   }
  },
  "serverInfo": {
   "name": "larva",
   "version": "0.8.0"
  }
 }
}
  • change: 1 is full synchronization: every didChange carries the whole text, and the server keeps only the last one. Documents are small enough that incremental edits would not pay for their complexity, and the analysis is whole-document anyway.
  • : triggers completion for .queen files (key: value); . triggers it for member access.
  • The token legend is built in Initialize from each pack's info request (tokenTypes), one pack after the other. tokenOffset remembers where each pack's types start, so two languages can share one legend.

Methods

Method Kind Handled by
initialize request larva: capabilities and token legend
initialized notification ignored
shutdown, exit request, notification larva: exit ends with 0 after shutdown, 1 without
textDocument/didOpen, didChange notification larva stores the text and runs Changed
textDocument/didSave notification a saved .queen is read from disk and every project reloads
textDocument/didClose notification lsp.close to the pack; the document's diagnostics are cleared
workspace/didChangeWatchedFiles notification Reload: projects read again, packs reset, open documents analyzed again
textDocument/completion request .queen: QueenFiles.Complete; else lsp.completion
textDocument/hover request .queen: QueenFiles.Hover; else lsp.hover
textDocument/definition request larva: document: larva from the decompiled links; else lsp.definition, then Definition turns it into a location
textDocument/signatureHelp request lsp.signatures
textDocument/documentSymbol request lsp.symbols
textDocument/semanticTokens/full request larva: document: larva from the decompiled tokens; else lsp.tokens, shifted by the pack's legend offset
workspace/symbol request lsp.workspaceSymbols of every pack in use
larva/decompile request (Larva's own) lsp.decompile: {type, language?, bodies?} → {text, uri}

larva/decompile backs the extension's Show Type (decompiled) command: it returns the text and the larva: URI of the file it was written to.

Language pack requests

A pack is a library loaded into larva's process (see Language packs). LspServer.Ask(pack, method, params) calls it with a JSON request and gets JSON back; a LanguagePackException is printed and treated as no answer, so a failing pack degrades one feature instead of ending the server.

Request Parameters Answer
lsp.analyze uri, path, text, others (the project's sources of this language), open (path → text of other open documents), workspace (key, options, references, foreign) the document's diagnostics, in LSP form
lsp.close uri —
lsp.reset — — (drops cached workspaces)
lsp.completion, lsp.hover, lsp.definition, lsp.signatures uri, line, character the LSP result (a definition is {file, line, character} or {decompiled, name})
lsp.symbols, lsp.tokens uri the outline, the token data
lsp.workspaceSymbols query hits with file or decompiled
lsp.decompile type, bodies, view, optionally assembly, arity, member {assembly, type, text, extension, tokens, links, at}

The pack keeps one workspace per workspace.key: the HIR catalog built from the references (System.Core, System.Bindings, the project's libraries). Building a catalog is the expensive part; the key lets every document of a project share one and makes a new one appear only when something it depends on changes. In the Antril pack, AntrilLanguage.Workspace caches them in a dictionary and lsp.reset clears it.

Inside the pack, the analysis of a document implements IDocumentAnalysis (Larva.Hir.Services/Editor): CompletionAt, SymbolAt, CallAt, Outline, Problems and Tokens. The language only says what is at a position: a CompletionContext (Members after value., Namespace after Some.Namespace., Scope for a name on its own, or Nothing inside a comment), a SymbolInfo, a CallSite. LanguageService turns that into lists and hovers using the catalog and the language's LanguageView, which writes signatures in the language's own syntax. Visibility (Visible), extension members (HirCatalog.ExtensionsOf, also through base classes, interfaces and constrained ext<T : IShape>) and auto-imports (CompletionItem.Import) are decided there once for all languages.

Projects

Every source file is analyzed as part of a program. ProjectOf(path) decides which:

  1. FindQueen walks up from the file's folder and takes the first folder with a *.queen file.
  2. Without one, the file is loose (Loose): its program is every source file in its folder, against System.Core (and System.Bindings), with this machine's defaults (LarvaDefaults.FrontendOptions, from larva config).
  3. With one, the project is read by QueenProject.Load, or by QueenProject.LoadText when the .queen is open in the editor, so unsaved edits count. Hive dependencies resolve through HiveResolver.

The walk stops at the first .queen found going up. A .queen file in a high folder (the home folder, for instance) makes every file below it part of that project, and the first analysis then compiles everything it lists.

A project becomes a Project record:

Field What it holds
Stamp The .queen path and write time (or open plus a hash of its text) and the stamp of larva's config file. A different stamp reads the project again.
Sources The project's sources and its same-GC dependencies' sources.
Assemblies, References System.Core, System.Bindings, and every compiled library the project names with assembly: (bindings, for instance).
Foreign Dependencies built for another gc:: their names and sources. The pack compiles them as references of their own domain, so a wrong crossing is reported where it is written (see GC domains).
Options QueenProject.FrontendOptions(): what the frontend is told, such as unsafe: false and the sources that deny unsafe code.
Key Stamp + options + each assembly's write time: the pack's workspace key. Rebuilding a bindings library makes a new key.
Settings Options, sources, assemblies, foreign dependencies: what changes analysis.

A .queen with errors does not stop analysis: the last project read is kept, and without one the file is analyzed as loose.

Analysis per change

Changed(uri, text) runs on every didOpen and didChange:

didChange(uri, text)
   │
   ├─ larva: URI ─────────▶ stored, nothing else (a decompiled file is not part of any program)
   ├─ .queen ─────────────▶ QueenChanged: QN diagnostics
   │                        ProjectEdited: open sources of that project re-analyzed if Settings changed
   └─ source file ────────▶ Analyze: lsp.analyze with the project → publishDiagnostics
                            BorrowCheckLater

Analyze sends the document's text plus the text of every other open document, so an unsaved change in one file is seen by the others. After analysis, the document's pack is remembered in analyzed; a request for a document not analyzed in its current state analyzes it first (PackOf).

Here is a document with one error. Opening it publishes:

Antril
import System;

into namespace Demo
{
	Main()
	{
		Console.WriteLine("hi");
		i32 x = "no";
	}
}
json
[
 {
  "range": {
   "start": {
    "line": 7,
    "character": 10
   },
   "end": {
    "line": 7,
    "character": 14
   }
  },
  "severity": 1,
  "code": "AN2084",
  "source": "antril",
  "message": "a `str` is not a `i32`"
 }
]

A hover on Console (line 6, character 4) answers with the declaration as the language writes it:

json
{
 "jsonrpc": "2.0",
 "id": 2,
 "result": {
  "contents": {
   "kind": "markdown",
   "value": "```antril\npublic static class Console\n```"
  }
 }
}

Reloading

Reload clears every cached project and System.Core, sends lsp.reset to each pack in use, and analyzes every open document again. It runs on workspace/didChangeWatchedFiles (the extension watches each language's source extensions and *.queen) and when a .queen is saved (didSave), whether or not the editor also reports it as a watched file. A project's settings (the borrow checker, unsafe code, the default access) therefore apply as soon as the file is saved, and edits to an open .queen apply even before.

.queen files

A .queen document is never sent to a language pack. QueenChanged parses its current text with QueenProject.LoadText and publishes the project reader's own diagnostics with source: "queen"; problems found in another file (an included project) are reported on the first line with that file's name and position.

QueenFiles (Larva.Cli/QueenFiles) answers completion and hover from QueenSchema: the chunk names, the keys of the chunk the cursor is in (the last chunk NAME: above it), the values a key takes, and an explanation of each key. For this text:

.queen
queen 1.0;
chunk PROJ:
   name: Demo

chunk DEPS:
   assembly: Missing.lva,
   

the server publishes

json
[{"range": {"start": {"line": 5, "character": 13}, "end": {"line": 5, "character": 25}}, "severity": 1, "code": "QN108", "source": "queen", "message": "`assembly` names a Larva library (`.dll` or `.lva`), not `Missing.lva`"}]

and completion on the empty line of DEPS offers that chunk's keys and chunk:

json
["package", "proj", "larvaobj", "assembly", "lib", "chunk"]

Decompiled documents

When a definition leads into a library (System.Core, a binding, a Hive package without sources), there is no source file to open. The pack decompiles the type instead, and larva shows it as a read-only document.

  1. lsp.definition answers {decompiled: {assembly, type, text, extension, tokens, links, at}, name}.
  2. Decompiled writes the text to %LOCALAPPDATA%\Larva\decompiled\<assembly>\<Type><extension> (only when it changed; $ in a type name becomes _) and remembers its tokens and links by path.
  3. Definition returns a location with the larva: scheme: the file's URI with file: replaced by larva: (DecompiledUri), at the offset at (the declaration of the exact member and overload), or at the first occurrence of the name when the pack gave none.

Asking for the definition of Console in the example above lands in System.Core\System.Console.ant under that folder, line 6, character 21.

The extension serves larva: URIs with a text document content provider that reads the file. The separate scheme matters: as a file: document, the decompiled type would be part of the folder's program and would exist twice, once in System.Core and once as a source, with every name a duplicate.

Inside a larva: document:

  • No analysis. Changed stores the text and returns; no diagnostics are published.
  • Colors come from the decompiler, not from an analysis. While a view writes a type (DecompileMarked in the Antril view), each name is wrapped in marks that record what it is; the marks are removed at the end and become DecompiledSpans with exact offsets. DecompiledTokens encodes them as LSP tokens, shifted by the language's legend offset.
  • Navigation uses the links: each carries the target type, its assembly and arity, and the member's Key. DecompiledDefinition finds the link under the cursor, decompiles the target, and jumps to its at. The assembly in the link keeps a program type with the same name as a library type from capturing the jump; the key selects the right overload. Keywords that name types (i32, str) are links without color.
  • After a restart, an editor may reopen a larva: document the new server has never seen. NamesOf then decompiles the type again from the file's name, taking the library from the folder name.

Semantic tokens

The pack computes tokens from the analysis (IDocumentAnalysis.Tokens: offset, length, SemanticKind, SemanticModifiers) and returns them in LSP's relative encoding: five integers per token (line delta, start delta, length, type index, modifier bits). larva only adds the pack's legend offset to each type index.

The kinds are the standard LSP token types, plus four of Larva's own that editors declare with a standard parent: field (parent property), exception and attribute (parent class), bitflags (parent enum). A theme that knows only the standard types still colors them. Semantic.Of assigns kinds from the catalog: a value is struct, a class derived from System.Exception is exception, a constant is variable with readonly and static, and a member of another assembly gets defaultLibrary. The server forces no colors; the editor's theme decides.

For the example document the answer is:

json
{"jsonrpc": "2.0", "id": 4, "result": {"data": [0, 7, 6, 0, 0, 2, 15, 4, 0, 1, 2, 1, 4, 11, 1, 2, 2, 7, 2, 8, 0, 8, 9, 12, 10, 1, 6, 1, 8, 1]}}
Tokens Position Text Type Modifiers
0, 7, 6, 0, 0 line 0, col 7 System 0 namespace —
2, 15, 4, 0, 1 line 2, col 15 Demo 0 namespace declaration
2, 1, 4, 11, 1 line 4, col 1 Main 11 function declaration
2, 2, 7, 2, 8 line 6, col 2 Console 2 class defaultLibrary
0, 8, 9, 12, 10 line 6, col 10 WriteLine 12 method static, defaultLibrary
1, 6, 1, 8, 1 line 7, col 6 x 8 variable declaration

Borrow check as you type

The borrow checker works on Larva IR, after lowering (see The borrow checker), so it cannot run inside a pack's analysis. larva runs it itself in BorrowCheckLater, after each analysis, when:

  • the project's borrowcheck (or the machine default from larva config) is enforce or warn;
  • the project has no foreign-GC dependencies;
  • the analysis found no errors in the document.

It takes a snapshot of the open documents, waits 400 ms on a background task, and gives up if another edit arrived meanwhile (borrowGeneration). Then BorrowErrors compiles the program as the editor has it with the pack's compile request, links it with System.Core and the project's assemblies (HirLinker.Link), lowers it with the project's GC (HirLowerer.Lower), and runs BorrowChecker.Check on each function whose source positions lie in the project's files: the libraries were checked when they were built. Each error is added to the diagnostics of the open document it belongs to, with source: "borrow checker", severity error for enforce and warning for warn, and the error kind as its code. If anything fails along the way, the borrow check publishes nothing and the analysis diagnostics stand.

The same checker runs on every larva build (not only --check), so the editor and the build agree.

URIs and paths

Editors escape URIs in different ways. VS Code sends file:///c%3A/Users/a%20b/x.ant, and the standard URI parser reads that as /c:/Users/a b/x.ant, which matched no .queen, no relative path and no decompiled file. LspServer.FilePath(rest) handles it:

  1. Unescape the whole path (%3A → :, %20 → space).
  2. //server/share/… (two slashes, not three) is a network path: on Windows \\server\share\….
  3. Otherwise collapse the leading slashes to one; on Windows, drop the slash before a drive letter (/c:/… → c:/…) and turn / into \.

PathOf applies it to file: URIs and DecompiledPath to larva: URIs. Paths sent back are built with new Uri(Path.GetFullPath(path)).AbsoluteUri, so the server always answers in one canonical form.

Binding profiles in the editor

A binding profile is an ordinary source file deriving from System.Bindings.BindingProfile (see Bindings internals). Core() loads System.Bindings with System.Core (the installed System.Bindings.dll, or the cached build of Library/System.Bindings in a working copy) and Libraries() adds it to the references of every project and loose file. A profile therefore gets completion, hover and errors with no special handling, and the language decides nothing about it.

Was this page helpful?

No address is stored; your user name is, if you are signed in.

Esc