Neuron Explorer
What is Neuron Explorer?
Neuron Explorer is an open-source tool for mechanistic interpretability research on the Qwen3.5-4B language model. It focuses on steering-based analysis, allowing users to activate or suppress individual multilayer perceptron neurons. The application loads the model locally using MPS, CUDA, or CPU backends and provides a web-based interface for live interaction.
The interface presents a three-dimensional visualization of the model architecture. Users navigate a map showing attention and MLP activity per layer, where hovering reveals steer and R lenses. A list view ranks neurons by word cluster mass and allows token-based lens searches. Each neuron entry displays steered rollouts, multi-turn chat comparisons, and activation contexts.
The tool includes automated sweep scripts that generate per-neuron activation statistics and run steering experiments across selected layers using a modified vLLM setup. Results are visualized through UMAP embeddings. An optional Claude integration acts as an MCP server agent, enabling natural language queries to explore neurons and run live inference.
Neuron Explorer features
- Interactive 3D visualization of model layers and neuron activity
- Live steering of MLP neurons with positive, negative, and neutral controls
- Automated sweep scripts for generating activation statistics and test runs
- UMAP embedding visualization of layer rollouts colored by neuron and prompt
- Multi-turn steered chat interface comparing different activation doses
- Optional Claude agent integration via Model Context Protocol for navigation
Who uses Neuron Explorer and for what
- Researchers analyzing how specific MLP neurons influence token generation in Qwen3.5-4B
- Developers testing the effect of activating or suppressing individual neurons on model outputs
- Teams running batch steering experiments across multiple layers to collect activation metrics
- Analysts visualizing neuron clustering and down-projection similarities using interactive maps
- Users exploring model internals through natural language queries powered by an integrated AI agent
Neuron Explorer FAQ
Is Neuron Explorer free?
Yes, the project is released under an MIT license and is available at no cost.
Does Neuron Explorer work on Mac?
The tool supports loading the Qwen3.5-4B model locally on macOS using the MPS backend, as well as CUDA and CPU environments.
What is Neuron Explorer used for?
It is used for mechanistic interpretability research, specifically for steering and visualizing multilayer perceptron neurons in the Qwen3.5-4B language model.
Is Neuron Explorer open source?
Yes, the source code is publicly available on GitHub under the MIT license.
Can I use Neuron Explorer with other models besides Qwen3.5-4B?
The website does not specify support for other models. The current version is built specifically for Qwen3.5-4B.
How do I install Neuron Explorer?
Installation requires Python and uv. You run uv sync to set up dependencies, then start the server with uv run python explorer/server.py.
Comments (0)
Comments and upvotes open when this product launches on ProductPerch.
More AI launches
Generate and edit images in plain language.
Create personalized AI portraits, images and videos.
Star in the viral Bulin 47 freestyle video from one photo
Try on 1000+ real hairstyles with realistic AI previews
Cinematic AI video from text, images, and audio
Turn a portrait into a 23-second AI party video
