I've been thinking of tightening up the supported Python versions in LibCST. See the issue [here](https://github.com/Instagram/LibCST/issues/776) if you want to comment. ## Problem Currently, LibCST allows for handling any Python version since 3.0. This is problematic on many fronts: - Maintaining the parser is a larger and larger burden going forward - Figuring out how to represent multiple versions within the CST is both challenging to implement and awkward to understand when using (e.g. "is this a `Decorator` node that supports any expression or just a few of them?") - Making sure a CST is valid for a certain Python version is non-trivial - No real motivation for fixing bugs and inaccuracies in old Python versions (good luck getting a PR about Python 3.3 support reviewed) So it's clear we should limit the amount of Python versions supported. Here are the two obvious (to me anyway) ways forward: ## Proposal 1: Strictly One Version In this world, a particular version of LibCST would only ever support one Python version. Whenever a new CPython gets released (currently at most once a year) with a new grammar, a new major release of LibCST will come out shortly after, dropping support for the previous grammar. - LibCST would start shipping codemods to future-proof code (e.g. `libcst==2.0` would have a codemod to transform code that `libcst==3.0` would be guaranteed to support) - (+) this is simple from the parsing point of view: if the targeted CPython parses your code, LibCST will, too. If the targeted CPython doesn't, LibCST won't either. - (+) simple to represent: the CST classes can more closely resemble the structure of the official `ast` module. It will be clear what can go into a `Decorator` node, given a particular LibCST version. There will never be a node representing the `print` statement. - (+) simple to validate: we could just round-trip the generated code through LibCST's own parser (this is probably impractical, but useful for testing the validation code) - long term, this option should motivate people to future-proof their codebases if they want to stay on the latest LibCST version - (-) tools using LibCST to parse Python code will likely have to pin the major version of LibCST or introduce some kind of facade to manage multiple versions - (-) we will likely have to maintain multiple concurrent versions of LibCST for the most actively used Python versions ## Proposal 2: Officially Supported CPython Versions In this world, only the actively supported Python versions (to be more precise, the actively supported CPython releases implementing a Python version) would work with LibCST. As of this writing (2022-09-15) that would mean 3.7-3.10: four versions. There will be a brief period of having to support 5 versions until 3.7 reaches end-of-life. Whenever a Python version is dropped from support, LibCST would bump its major release number. This is a less pure, more practical version of proposal 1, that compromises on complexity: the grammar, validation rules, and CST nodes would be more complex and ambiguous (user code will have to make sure they only put the correct type of expression into a `Decorator`, or avoid using a `Name` that becomes a keyword in later Python versions). But at least tools using LibCST will have a clearer path forward. The complexity in the grammar can be solved by shipping multiple copies of the parser, each being capable of parsing a single Python version. This means that parsing would have to try each grammar in sequence, unless a requirement for a particular version is passed in (#[740](https://github.com/Instagram/LibCST/issues/740)) I don't see an easy way to reduce the complexity on the representation and validation side without hurting ergonomy. ## Conclusion I don't have a conclusion yet. I'm slightly leaning towards Proposal 1, but am open to be convinced otherwise. See the issue [here](https://github.com/Instagram/LibCST/issues/776) if you want to comment.