Guerrilla Coding: Enabling subcultures to thrive through custom made tooling

I’ve been working in tech for fifteen years and over a decade of this has been as a software engineer. Across all companies and diverse projects I worked on, ranging from a custom database as a service running on a mix of private and public cloud and payment orchestration for the venture capital ecosystem, I noticed that companies will naturally cultivate their own engineering culture. Even deeper, many organizations within a company or even teams might adapt the broader guidelines into their own subculture of code standards, project rituals and principles valued. In my observations it is clear that smaller companies, orgs, teams tend to have more cohesion on the shared standards being upheld and as an organization grows, subcultures also grow – this behavior even impacts individuals as there is a tendency for people to relate more to the subculture they are helping create than to the complexity of the broader organization.

The formation of these tribes might be influenced by several factors, range of experience, different tech stack (front end, back end, infra, programming languages, etc), line of business requirements or simply personal alignment. An example of this is the creation of a [front end, back end] "guild” where individuals meet to talk about how to manage technology related to their interests. These groups will then specify what "code quality” means and elect tools to help them deliver projects, IDE plugins, linters, automated tests, manual tests, continuous integration(CI)/delivery(CD), etc. In my experience, those tools work perfectly if we are able to customize their behavior to how our subculture operates but, as we tailor our development workflow, remember that available tools may only cover a fraction of the checks needed. To maintain quality, the organization must establish robust frameworks and linters. This effort is vital in ensuring their standards are met, transforming potential pitfalls into pathways of excellence. Quality should be a shared commitment, woven into every aspect of the process. React, Nebula and Black are examples of a library, a collection of plugins and a code formatter used by thousands of teams around the world.

Ok, but where are you going?

In some python projects where I applied hexagonal architecture I have experimented with this python library that facilitates dependency injection(DI), and yes I know python is not java 😛. We take DI and auto wiring for granted on spring boot and are not even aware how spoiled we are. Hexagonal architecture aims to separate a system's core logic from its external integrations. This means business logic doesn't rely on specific data models, ORM, or data transfer methods like HTTP or gRPC. Instead, it depends on domain objects, such as Python dataclasses or Plain Old Java Objects (POJOs), and contracts, which can take the form of interfaces, abstract base classes, or Python's structural subtyping.

Hexagonal architecture illustration

A simplistic implementation of the hexagonal architecture using python and the aforementioned DI library follows:

❯ tree car_application/
car_application/
├── car_application
│   ├── __init__.py
│   └── adapters
│       ├── __init__.py
│       └── out
│           ├── __init__.py
│           └── car_repository.py
├── container.py
└── domain
    ├── __init__.py
    ├── application
    │   ├── __init__.py
    │   └── command_handlers
    │       ├── __init__.py
    │       └── create_car.py
    ├── entities
    │   ├── __init__.py
    │   ├── car.py
    │   └── make.py
    └── ports
        ├── __init__.py
        └── car_repository.py

# file: car_application/domain/entities/make.py
class Make(Enum):
    VOLVO = "VOLVO"
    HONDA = "HONDA"
    ...

# file: car_application/domain/entities/car.py
from car_application.domain.entities.make import Make


@dataclass
class Car:
    id: int | None
    make: Make
    model: str
    year: int

# file: car_application/domain/ports/car_repository.py
from car_application.domain.entities.car import Car


class CarRepository(ABC):
    @abstractmethod
    def exists(self, id: int) -> bool:
        ...
    
    @abstractmethod
    def save(self, car: Car) -> None:
        ...

# file: car_application/domain/application/command_handlers/create_car.py
from car_application.domain.ports.car_repository import CarRepository
from car_application.domain.entities.car import Car


class CreateCarCommandHandler:
    def __init__(self, car_repository: CarRepository):
        self.car_repository = car_repository
    
    def handle(self, car: Car) -> None: # not using a Command param for brevity
        if car.id and self.car_repository.exists(car.id):
            return
        # complex business logic goes here and eventually save the car in the DB
        self.car_repository.save(car)

# file: car_application/adapters/out/car_repository.py
from car_application.domain.ports.car_repository import CarRepository
from car_application.domain.entities.car import Car
from car_application.models import Car as CarModel # Some data model in Django


class CarRepository(CarRepository):
    def __init__(self, car_model: Type[CarModel]) -> None:
        self.car_model = car_model

    def exists(self, id: int) -> bool:
        return self.car_model.filter(id=car.id).exists()
    
    def save(self, car: Car) -> None:
        car_instance: CarModel = Mapper.transform(car).into(CarModel, using=Strategy.AttributeNamesMatch) # serializer/adapter/mapper omitted for brevity
        self.car_model.create(car_instance)

# file: car_application/container.py
from dependencies import Injector

from car_application.adapters.out.car_repository import CarRepository # notice we are importing the implementation
from car_application.domain.application.command_handlers.create_car import CreateCarCommandHandler # notice we import the business logic from the domain as it does not depend on external logic
from car_application.models import Car as CarModel # Some data model in Django


class AppContainer(Injector): # yes, unfortunately components are not wired automatically yet
    car_model = CarModel
    car_repository = CarRepository 
    create_car_command_handler = CreateCarCommandHandler

# Then in your DRF, gRPC, celery task, kafka event handler you can
from car_application.container import AppContainer
from car_application.domain.entities.car import Car


def handle_event(car: ExternalRepresentationOfACar) -> None:
    car_domain: Car = Mapper.transform(car).into(Car, using=Strategy.AttributeNamesMatch) # serializer/adapter/mapper omitted for brevity
    AppContainer.create_car_command_handler.handle(car_domain)

If you have created code in python, maybe like this or even using “service” classes, that uses DI and where components do not explicitly depend on a specific implementation you have had trouble navigating the codebase. Probably you’ve heard comments like, “it is hard navigating the codebase”, “there are multiple components with a similar name”, “when we look for implementations on our IDE it gives us a list as opposed to being able to navigate directly to the implementation we are using”. Looking closely, this is the navigation workflow that people usually experience:

• Engineer navigates to car_application/domain/application/command_handlers/create_car.py
• Engineer “goes to symbol” CarRepository, the IDE using an LSP takes them to the interface
• Engineer then needs to find implementations of the CarRepository
  • Engineer gets frustrated there are multiple implementations and wastes time navigating the codebase
• Engineer eventually finds the container.py file and the implementation being injected, then they have to go to the implementation
• The cycle repeats many times for different components
• Engineer gets frustrated and eventually learns how to navigate the codebase

Navigating code like this is a waste of effort, especially for engineers being onboarded, and can lead to hundreds of hours dissipated yearly. As we think about the context of this post, this is one of the points where groups might introduce customized tooling and shared standards to help engineers be more effective. And while I recognize there can be multiple possible solutions to “navigating codebases effectively”, where some might be as simple as using explicit and direct references to components, I wanted to explore it from a perspective I have never explored before: creating a custom Language Server Protocol(LSP) server that expects a codebase to contain container.py files, parse these files and use the attributes in the classes that inherit from Injector to support “Go to implementation” queries. Yes, it is super customized and that is exactly my point. The navigation workflow would work like this:

• Engineer navigates to car_application/domain/application/command_handlers/create_car.py on their IDE
• Engineer goes to the car_repository: CarRepository and issues a “Go to implementation” query
• The LSP finds car_application.container.AppContainer.car_repository and responds with its location
• The IDE handles the LSP response and opens the location that was returned

Let's get to it!

LSP implementation

This is a proof-of-concept(POC) and I will focus on doing the smallest amount of work that helps me validate my idea, so I won’t be concerned with anything other than “works on my machine” 😛. With that in mind, I will use Python, pygls and lsprotocol to create the language server and will implement a simple neovim plugin to be able to issue the LSP queries and test the IDE experience.

LSP server code

import ast
import functools
import logging
import os
from urllib.request import pathname2url, url2pathname

from lsprotocol.types import (
    Location,
    Position,
    Range,
    TextDocumentPositionParams,
)
from pygls.server import LanguageServer

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
    handlers=[logging.StreamHandler()],
)
logger = logging.getLogger("pygls")
logger.setLevel(logging.INFO)
CONTAINER_FILE_NAME: str = "container.py"


class DependencyInjectionGoToImplementation:
    def find_implementations(self, symbol_name: str, root_path: str):
        """
        Search for the implementations (container attributes) that match the symbol_name.
        Returns locations only for matching attributes in the container file.
        """
        logger.info(f"Root path: {root_path}")

        # Search for all CONTAINER_FILE_NAME files starting from the root
        container_file_paths = self.find_container_files(root_path)
        logger.info(f"Found {CONTAINER_FILE_NAME} files: {container_file_paths}")

        if not container_file_paths:
            return []

        implementations = []
        for container_file_path in container_file_paths:
            tree = self.parse_container_file(container_file_path)
            logger.info(f"Parsing container file: {container_file_path}")

            classes, attributes = self.extract_symbols_from_ast(
                tree, container_file_path
            )
            logger.info(f"Classes: {classes}, Attributes: {attributes}")

            # Only add the location if the attribute name matches the symbol being searched
            for attribute in attributes:
                if attribute["name"] == symbol_name:
                    location = Location(
                        uri=file_path_to_uri(
                            attribute["container_file"]
                        ),  # Use container file and convert to file URI
                        range=Range(
                            start=Position(
                                line=attribute["line"] - 1,
                                character=attribute["col_offset"],
                            ),
                            end=Position(
                                line=attribute["line"] - 1,
                                character=attribute["col_offset"]
                                + len(attribute["name"]),
                            ),
                        ),
                    )
                    implementations.append(location)

        return implementations

    def find_container_files(self, root_path):
        container_files = []
        for root, _, files in os.walk(root_path):
            for file in files:
                if file == CONTAINER_FILE_NAME:
                    container_files.append(os.path.join(root, file))
        return container_files

    def parse_container_file(self, container_file_path):
        with open(container_file_path, "r") as file:
            tree = ast.parse(file.read(), filename=container_file_path)
        return tree

    def extract_symbols_from_ast(self, tree, container_file_path):
        """Extract class names and assigned attributes (dependencies) from an AST."""
        classes = []
        attributes = []

        for node in ast.walk(tree):
            if isinstance(node, ast.ClassDef):
                classes.append(node.name)

                # Check if the class inherits from Injector (or similar)
                if any(base.id == "Injector" for base in node.bases):
                    # After finding the class, look for assignments within the class body
                    for class_node in node.body:
                        if isinstance(class_node, ast.Assign):
                            for target in class_node.targets:
                                if isinstance(target, ast.Name):
                                    # Save the attribute name and the line/column where it is assigned
                                    attributes.append(
                                        {
                                            "name": target.id,
                                            "line": class_node.lineno,
                                            "col_offset": class_node.col_offset,
                                            "container_file": container_file_path,
                                        }
                                    )

        return classes, attributes


# LANGUAGE SERVER
server = LanguageServer(name="di_lsp", version="0.0.1")


@server.feature("textDocument/implementation")
def find_implementations(ls: LanguageServer, params: TextDocumentPositionParams):
    # Extract the file URI and position
    uri = params.text_document.uri
    position = params.position

    # Read the file contents (from URI) and get the word at the position
    document = ls.workspace.get_document(uri)  # Get document based on URI
    line = document.lines[position.line]  # Get the line of text at the position
    symbol_name = get_word_at_position(
        line, position.character
    )  # Get the word at the character position

    logger.info(f"Implementation Symbol name: {symbol_name}")
    logger.info(f"params: {params}")

    go_to_implementation: DependencyInjectionGoToImplementation = (
        DependencyInjectionGoToImplementation()
    )
    return go_to_implementation.find_implementations(
        symbol_name, root_path=ls.workspace.root_path or get_root_path(uri)
    )


# HELPERS
@functools.lru_cache(maxsize=256)
def file_path_to_uri(file_path):
    # Convert file path to file:// URI
    return f"file://{pathname2url(file_path)}"


def get_word_at_position(line: str, character: int) -> str:
    """
    Utility function to extract the word at the given character position
    on a specific line. Words are defined as alphanumeric characters and underscores.
    """
    start = character
    end = character

    # Move backward to find the start of the word (including underscores)
    while start > 0 and (line[start - 1].isalnum() or line[start - 1] == "_"):
        start -= 1

    # Move forward to find the end of the word (including underscores)
    while end < len(line) and (line[end].isalnum() or line[end] == "_"):
        end += 1

    # Extract and return the word
    return line[start:end]


@functools.lru_cache(maxsize=256)
def get_root_path(uri: str) -> str:
    # Convert the URI to a path (strip the "file://" part)
    file_path = url2pathname(uri)

    # Try to find the project root (you could customize this to check other files like pyproject.toml, etc.)
    while file_path:
        if os.path.isdir(os.path.join(file_path, ".git")):
            return file_path  # Return the path of the Git repository root
        parent_path = os.path.dirname(file_path)

        if parent_path == file_path:  # Reached the filesystem root
            break

        file_path = parent_path

    return os.path.dirname(
        uri
    )  # Default to the current file's directory if no root is found


if __name__ == "__main__":
    logger.info("Starting language server")
    server.start_io()

Neovim plugin

❯ tree plugins/
plugins/
└── neovim
    └── di-ls-plugin
        ├── lua
        │   └── di_lsp
        │       ├── doc
        │       │   ├── di_lsp.txt
        │       │   └── tags
        │       └── init.lua
        └── plugin
            ├── di_lsp.txt
            └── di_lsp.vim

This is the tree of neovim plugin directory, the text files are supposed to contain documentation but I couldn’t get neovim to actually parse it and instrument it through :h di_lsp

" di_lsp.vim
lua require('di_lsp').setup()

The plugin depends on lspconfig being installed.

-- init.lua
-- nvim_lsp setup for DI LSP (a custom language server)
-- This script configures the DI LSP server to work with Neovim's built-in LSP client.

-- Import the LSP configuration module from Neovim
local nvim_lsp = require('lspconfig')

-- Module table to encapsulate the setup function
local M = {}

-- Setup function to configure the DI LSP server
-- @param opts: A table of optional configurations
--   opts.cmd: The command to start the language server (default is a Python virtual environment)
--   opts.filetypes: A list of filetypes that the LSP server should handle (default is "python")
--   opts.root_dir: The root directory pattern used to detect the project root (default looks for ".git" or "container.py")
--   opts.settings: Custom settings for the LSP server (default is empty)
--   opts.on_attach: A function that is called when the LSP server attaches to a buffer
function M.setup(opts)
    -- Ensure the options table is not nil and defaults are applied if necessary
    opts = opts or {}

    -- Configure the DI LSP server
    nvim_lsp.di_lsp.setup {
        -- Command to launch the DI LSP server
        -- Default is a Python virtual environment and a specific Python script
        cmd = opts.cmd or {
            "<path/to/virtualenv/python/with/dependencies/installed>/python",
            "<path/to/where/the/script/is/located>/language_server.py"
        },

        -- Filetypes that the server should support (default is "python")
        filetypes = opts.filetypes or { "python" },

        -- Root directory detection logic, default is looking for ".git" or "container.py"
        root_dir = nvim_lsp.util.root_pattern(".git", "container.py"),

        -- Custom settings for the LSP server (default is an empty table)
        settings = opts.settings or {},

        -- Function to be called when the LSP server attaches to a buffer
        -- Here you can add additional logic such as keybindings or custom messages
        on_attach = function(client, bufnr)
            -- Print a success message when the LSP is successfully started
            print("DI LSP started 🚀! Stop crying and start coding! 🤓")
            -- Additional actions can be added here (e.g., key mappings, etc.)
        end,
    }
end

-- Return the module table to be used by other parts of the configuration
return M

Installing the neovim plugin

The neovim plugin installation depends on vim-plug, mason and lspconfig being available:

Plug '<path/to>/plugins/neovim/di-ls-plugin/'

Load the library within a lua block:

local lspconfig = require 'lspconfig'
local configs = require 'lspconfig.configs'

if not configs.di_lsp then
  configs.di_lsp = {
    default_config = {
      cmd = { 'python', '<path/to>/language_server.py'},
      root_dir = lspconfig.util.root_pattern('.git'),
      filetypes = { 'python' },
    },
  }
end
lspconfig.di_lsp.setup {}

require("mason-lspconfig").setup_handlers({
  -- Add a custom handler for the di_lsp server
  ["di_lsp"] = function()
    require("lspconfig").di_lsp.setup {
    on_attach = on_attach,
    capabilities = capabilities,
    flags = {
      debounce_text_changes = 150,
      allow_incremental_sync = true,
    },
    on_init = function(client)
      client.config.settings.diagnosticMode = "openFilesOnly"
      client.config.settings.useLibraryCodeForTypes = true
      client.config.settings.autoSearchPaths = false
      client.config.settings.autoImportCompletions = true
      return true
    end,
    }
  end,
})

Voila!

The LSP integration with neovim is remarkable. I got it working in a couple of hours, despite no prior experience with LSPs or creating editor plugins.

Demo

It actually works! I did an interesting exploration with some spare time, and it's a basic idea that didn't take a lot of thought. However, I can see this evolving into a respectable LSP with actual plugins for popular IDEs.

Conclusion

I believe this POC proves that tooling can be built to improve the developer experience of python codebases and that tailor made LSP servers are a viable alternative to helping organizations be more effective. Here are some use cases that could benefit from a custom LSP:

• A domain-specific language (DSL) that is only used at your company
• Help inform declarative based systems using YAML or JSON of the directives supported by the system (e.g: https://github.com/mrjosh/helm-ls)
• Help aid code migration for collection of fields that are not fully known on application code and might be retrieved from a database
• Custom cross project type sharing through shared interfaces where protobuf or similar is not a viable option