← Research

Provenance of LLM-generated code

Code models sometimes reproduce snippets from their training data without saying where they come from. We find the source by shortlisting candidates with vector search and re-ranking them with code fingerprints.

When a reproduced snippet comes from code under a license, using it without knowing the source can mean plagiarism or a license violation. Fingerprinting tools such as Winnowing find the source accurately, but they compare the snippet with every document in the corpus, so their cost grows linearly with it. That is too slow when the corpus holds billions of snippets.

The snippet is embedded, vector search keeps its nearest neighbours in the corpus (100 in the paper, 8 here), and Winnowing fingerprints re-rank them. The true source was only third closest in the embedding, and comes first once fingerprints are compared.

At Télécom Paris, with Andrea Gurioli, Federico Pennino, Maurizio Gabbrielli and Stefano Zacchiroli, we trained SourceTracker, an encoder that maps code to vectors so that a fragment lands near the snippet it came from, and combined it with Winnowing in HybridSourceTracker. Vector search is fast but approximate, so it only builds a shortlist. Fingerprints are exact but slow, so they only compare the fragment with that shortlist. Once a fragment is a few lines long, the pair finds the source more often than fingerprints alone, and the search stays fast as the corpus grows.

Paper