# GitHub resources for Data Science & AI

_22 thoughts, 20 connections_

### 10 GitHub repos every Data Scientist should bookmark in 2026
Tags: #data-science #ai-engineering #github
Priority: ★★★★★
Connected: enables "rasbt/LLMs-from-scratch", enables "karpathy/nn-zero-to-hero", enables "mlabonne/llm-course", enables "DataTalksClub/llm-zoomcamp", enables "microsoft/generative-ai-for-beginners", enables "Shubhamsaboo/awesome-llm-apps", enables "GokuMohandas/Made-With-ML", enables "vllm-project/vllm", enables "langchain-ai/langgraph", enables "dair-ai/Prompt-Engineering-Guide"

A practical learning map for Data Scientists whose role is expanding into LLMs, RAG, evaluation, serving, MLOps, and agents.

The point is not to become a full AI Engineer overnight. It is to understand more of the modern stack so you can move confidently from notebooks and models into production AI systems.

Learning path:

Foundations & model internals
→ RAG and application building
→ Production / MLOps / serving
→ Agents and context engineering

Use this map as a guided reading and project list. Each child node explains what the repository teaches, why it matters for a Data Scientist, what to focus on first, and a practical way to learn from it.

### rasbt/LLMs-from-scratch
Tags: #foundations #llm #pytorch
Connected: depends on "10 GitHub repos every Data Scientist should bookmark in 2026"

Focus: Foundations

What it teaches:
• Build a GPT-style large language model step by step in PyTorch.
• Tokenization, embeddings, attention, transformer blocks, pretraining, and fine-tuning.
• How the pieces of an LLM fit together instead of treating the model as a black box.

Why it matters for a Data Scientist:
Modern Data Science increasingly involves LLM-based systems. You do not need to train foundation models yourself, but understanding their internals makes debugging, evaluation, and model selection much easier.

What to focus on first:
1. Tokenization and embeddings
2. Self-attention
3. Transformer blocks
4. Training loop
5. Fine-tuning

Practical project:
Build the smallest working GPT-style model you can, train it on a tiny text corpus, then change one component and observe what happens.

Best use: Deep understanding, not quick API usage.

Source:
https://github.com/rasbt/LLMs-from-scratch

### karpathy/nn-zero-to-hero
Tags: #deep-learning #foundations #neural-nets
Connected: depends on "10 GitHub repos every Data Scientist should bookmark in 2026"

Focus: Deep Learning Core

What it teaches:
• Neural networks from first principles.
• Backpropagation, gradients, optimization, language modeling, and progressively deeper architectures.
• How frameworks automate the math you need to understand when models behave badly.

Why it matters for a Data Scientist:
LLMs sit on top of deep-learning fundamentals. This repo strengthens the mental models needed to diagnose training instability, overfitting, bad gradients, and unexpected model behavior.

What to focus on first:
1. Micrograd and backpropagation
2. Makemore
3. Embeddings and hidden layers
4. Training dynamics
5. Transformer intuition

Practical project:
Implement a tiny neural network without hiding behind high-level libraries, then reproduce the same model in PyTorch and compare the two.

Best use: Rebuilding intuition when deep learning has become too abstract.

Source:
https://github.com/karpathy/nn-zero-to-hero

### mlabonne/llm-course
Tags: #fine-tuning #llm #evaluation
Connected: depends on "10 GitHub repos every Data Scientist should bookmark in 2026"

Focus: Model Specialization

What it teaches:
• The practical LLM Engineer roadmap.
• Fine-tuning with LoRA / QLoRA.
• Quantization approaches such as GGUF / AWQ.
• Dataset preparation, evaluation, and model specialization.

Why it matters for a Data Scientist:
Once you move beyond prompting, you need to know when to customize a model, how to do it efficiently, and how to measure whether the specialized model is actually better.

What to focus on first:
1. When fine-tuning is justified
2. LoRA / QLoRA intuition
3. Dataset quality
4. Quantization trade-offs
5. Evaluation after specialization

Practical project:
Take a small open model and specialize it on a narrow domain dataset. Compare the base model and tuned model on a fixed evaluation set.

Best use: Moving from model user to model customizer.

Source:
https://github.com/mlabonne/llm-course

### DataTalksClub/llm-zoomcamp
Tags: #rag #production #retrieval
Connected: depends on "10 GitHub repos every Data Scientist should bookmark in 2026"

Focus: RAG & Production

What it teaches:
• Practical LLM application engineering.
• Vector databases and semantic search.
• Hybrid search and retrieval-augmented generation.
• Evaluation of RAG systems.
• Containerized deployment and production thinking.

Why it matters for a Data Scientist:
RAG is one of the most common ways Data Scientists turn company knowledge into useful AI applications. The hard part is not calling the model. It is retrieval quality, evaluation, and deployment.

What to focus on first:
1. Embeddings and vector search
2. Chunking and retrieval quality
3. Hybrid search
4. RAG evaluation
5. Deployment

Practical project:
Build a RAG assistant over a small set of technical documents. Create 20 test questions and measure whether retrieval and answers improve as you change chunking and search settings.

Best use: Learning how to move from LLM demo to usable application.

Source:
https://github.com/DataTalksClub/llm-zoomcamp

### microsoft/generative-ai-for-beginners
Tags: #genai #full-stack #agents
Connected: depends on "10 GitHub repos every Data Scientist should bookmark in 2026"

Focus: Full-Stack Generative AI

What it teaches:
• A structured path from basic generative AI concepts to practical application building.
• API integration, structured outputs, function calling, UX patterns, and lightweight agents.
• How model capabilities connect to actual product experiences.

Why it matters for a Data Scientist:
Many Data Scientists know modeling better than application design. This repo helps bridge the gap between model output and a usable AI feature inside a real workflow.

What to focus on first:
1. Model/API interaction
2. Structured outputs
3. Function calling
4. UX patterns for AI
5. Lightweight agent behavior

Practical project:
Build a small assistant that reads a user request, returns structured JSON, calls one tool, and explains the result in plain language.

Best use: Understanding the full application layer around a model.

Source:
https://github.com/microsoft/generative-ai-for-beginners

### Shubhamsaboo/awesome-llm-apps
Tags: #architectures #llm-apps #multimodal
Connected: depends on "10 GitHub repos every Data Scientist should bookmark in 2026"

Focus: Reference Architectures

What it teaches:
• Ready-to-run examples of modern LLM applications.
• RAG patterns, multimodal apps, voice workflows, local-model integrations, and agentic systems.
• How different components are assembled into complete applications.

Why it matters for a Data Scientist:
Sometimes the fastest way to learn is to inspect a working architecture. This repo gives you patterns you can compare, adapt, and critique instead of starting every AI application from a blank page.

What to focus on first:
1. Pick one architecture close to your work
2. Trace the data flow end to end
3. Identify where retrieval, tools, memory, and models sit
4. Understand the evaluation points
5. Rebuild a simplified version

Practical project:
Choose one reference app and recreate only its essential path with your own dataset. Document which components were necessary and which were optional.

Best use: Architecture inspiration and rapid experimentation.

Source:
https://github.com/Shubhamsaboo/awesome-llm-apps

### GokuMohandas/Made-With-ML
Tags: #mlops #production #systems
Connected: depends on "10 GitHub repos every Data Scientist should bookmark in 2026"

Focus: MLOps & Systems

What it teaches:
• End-to-end production machine learning engineering.
• Data pipelines, testing, CI/CD, serving, monitoring, and reproducibility.
• How to structure ML work so it survives beyond a notebook or one-off experiment.

Why it matters for a Data Scientist:
A model has little value if nobody can reliably use, update, monitor, or reproduce it. This repo helps close the gap between experimentation and a maintainable production system.

What to focus on first:
1. Project structure
2. Testing ML code and data
3. Pipelines and reproducibility
4. Deployment patterns
5. Monitoring and iteration

Practical project:
Take one existing notebook project and convert it into a small reproducible ML service with tests, a training pipeline, versioned artifacts, and basic monitoring.

Best use: Learning what happens after the notebook.

Source:
https://github.com/GokuMohandas/Made-With-ML

### vllm-project/vllm
Tags: #serving #inference #llm
Connected: depends on "10 GitHub repos every Data Scientist should bookmark in 2026"

Focus: High-Throughput Serving

What it teaches:
• How modern LLM inference engines serve many requests efficiently.
• PagedAttention, continuous batching, KV-cache management, and throughput optimization.
• The systems trade-offs behind latency, memory use, and concurrency.

Why it matters for a Data Scientist:
Once an LLM application reaches real users, inference cost and speed become part of the product. Understanding serving helps you make better model and architecture choices instead of treating deployment as somebody else's problem.

What to focus on first:
1. Why inference is expensive
2. KV cache intuition
3. Continuous batching
4. Memory vs throughput trade-offs
5. Serving benchmarks

Practical project:
Serve the same small open model with a basic setup and with vLLM. Compare latency and throughput under multiple concurrent requests.

Best use: Understanding the systems layer behind production LLMs.

Source:
https://github.com/vllm-project/vllm

### langchain-ai/langgraph
Tags: #agents #orchestration #langgraph
Connected: depends on "10 GitHub repos every Data Scientist should bookmark in 2026"

Focus: Agent Orchestration

What it teaches:
• How to build stateful, cyclic AI workflows instead of simple one-shot chains.
• Agent state, branching, tool use, persistence, checkpoints, and human-in-the-loop controls.
• How to design systems where an AI can take multiple steps while remaining observable and controllable.

Why it matters for a Data Scientist:
As AI work moves from prediction to action, Data Scientists increasingly need to understand workflow design, state, failure handling, and evaluation across multiple steps.

What to focus on first:
1. State and graph structure
2. Nodes and conditional edges
3. Tool calling
4. Persistence and checkpoints
5. Human approval points

Practical project:
Build a small research agent that searches, summarizes, checks its own result, and pauses for human approval before producing a final recommendation.

Best use: Learning how agent systems are orchestrated beyond a single prompt.

Source:
https://github.com/langchain-ai/langgraph

### dair-ai/Prompt-Engineering-Guide
Tags: #prompting #context #security
Connected: depends on "10 GitHub repos every Data Scientist should bookmark in 2026"

Focus: Context Engineering

What it teaches:
• Prompting patterns from simple zero-shot instructions to more structured reasoning and tool-oriented patterns.
• ReAct-style workflows, grounding, context design, and defensive prompting.
• How prompt quality, context quality, and system constraints interact.

Why it matters for a Data Scientist:
Prompting is not just writing clever instructions. In production AI systems, the bigger skill is designing the right context, examples, constraints, retrieved information, and evaluation around the model.

What to focus on first:
1. Clear task instructions
2. Few-shot examples
3. Grounding with external context
4. ReAct and tool use
5. Prompt injection and defensive design

Practical project:
Take one recurring analytical task and create three versions: a basic prompt, a grounded prompt with reference data, and a tool-using workflow. Compare accuracy and failure modes.

Best use: Moving from prompt writing to context engineering.

Source:
https://github.com/dair-ai/Prompt-Engineering-Guide

### 10 GitHub repos for local LLM inference
Tags: #local-llm #inference #data-science
Priority: ★★★★★
Connected: enables "ggerganov/llama.cpp + llama-cpp-python", enables "vllm-project/vllm", enables "sgl-project/sglang", enables "turboderp/exllamav2", enables "ml-explore/mlx-lm", enables "NVIDIA/TensorRT-LLM", enables "dottxt-ai/outlines", enables "ollama/ollama", enables "exo-explore/exo", enables "flashinfer-ai/flashinfer"

A practical map for Data Scientists choosing tools to run LLMs locally or on their own hardware.

When evaluating local inference tools, the most important trade-offs are usually:
• latency: time-to-first-token and decode speed
• memory footprint: quantization and KV-cache efficiency
• Python integration: ease of use in notebooks, scripts, and batch jobs
• output control: structured, deterministic, schema-constrained generation
• hardware fit: CPU, NVIDIA GPU, Apple Silicon, or multi-device setups

Ecosystem map:
CPU & hybrid RAM → llama.cpp, llama-cpp-python, Ollama
NVIDIA GPU engines → vLLM, SGLang, ExLlamaV2, TensorRT-LLM
Apple Silicon / distributed → MLX-LM, Exo
Inference control / kernels → Outlines, FlashInfer

Use this map to choose by workload rather than hype. Each child node explains what the tool is, why it matters for Data Science inference, what to focus on first, and when it is the best fit.

### ggerganov/llama.cpp + llama-cpp-python
Tags: #llama.cpp #gguf #local-inference
Connected: depends on "10 GitHub repos for local LLM inference"

Focus: CPU / hybrid local inference

What it is:
llama.cpp is the foundational C/C++ engine for running GGUF-quantized LLMs across CPUs and consumer GPUs through Metal, CUDA, and ROCm. llama-cpp-python adds Python bindings for direct use from notebooks and scripts.

Why it matters for Data Scientists:
It gives unusually fine control over local inference without requiring a separate web service. You can tune GPU offloading, CPU threads, sampling, grammar-constrained decoding, and memory use while keeping inference in-process.

What to focus on first:
1. GGUF quantization
2. n_gpu_layers and CPU/GPU offloading
3. context size and memory trade-offs
4. sampling controls
5. grammar / structured decoding

Best fit:
When you want maximum portability and control across laptops, desktops, CPUs, Apple Silicon, or mixed hardware.

Practical project:
Run the same model at two quantization levels and compare memory use, latency, and output quality from a Jupyter notebook.

Sources:
https://github.com/ggerganov/llama.cpp
https://github.com/abetlen/llama-cpp-python

### vllm-project/vllm
Tags: #vllm #gpu #batch-inference
Connected: depends on "10 GitHub repos for local LLM inference"

Focus: high-throughput GPU inference

What it is:
A Python/C++ inference engine built around PagedAttention and continuous batching.

Why it matters for Data Scientists:
It is useful not only for serving APIs but also for offline batch inference. The native Python API makes it practical for scoring large datasets without HTTP overhead.

What to focus on first:
1. PagedAttention intuition
2. continuous batching
3. KV-cache efficiency
4. SamplingParams
5. offline batch inference

Best fit:
Large batch workloads on GPUs where throughput and memory efficiency matter.

Practical project:
Score a dataframe with thousands of prompts using the Python API and compare throughput with a naive Transformers loop.

Source:
https://github.com/vllm-project/vllm

### sgl-project/sglang
Tags: #sglang #radixattention #inference
Connected: depends on "10 GitHub repos for local LLM inference"

Focus: fast structured and repeated-prompt inference

What it is:
An inference and programming system designed for complex prompts, multi-turn workflows, structured generation, and efficient reuse of prompt prefixes.

Why it matters for Data Scientists:
RadixAttention can reuse KV states across repeated prefixes, which is valuable for synthetic-data generation, few-shot experiments, evaluation loops, and agent workflows.

What to focus on first:
1. RadixAttention
2. repeated-prefix caching
3. structured decoding
4. multi-turn workflows
5. throughput benchmarking

Best fit:
Workloads with many related prompts, repeated context, or agentic loops.

Practical project:
Run a few-shot classification pipeline across thousands of rows and compare performance with and without prefix reuse.

Source:
https://github.com/sgl-project/sglang

### turboderp/exllamav2
Tags: #exllamav2 #nvidia #quantization
Connected: depends on "10 GitHub repos for local LLM inference"
Source: https://github.com/turboderp-org/exllamav2 (verified 2026-09-10)

Focus: fast single-stream NVIDIA inference

What it is:
A specialized C++/CUDA inference engine tuned for NVIDIA GPUs and EXL2/GPTQ quantized models.

Why it matters for Data Scientists:
It is useful when you care about fast local token generation on consumer GPUs and need aggressive quantization to fit larger models into limited VRAM.

What to focus on first:
1. EXL2 quantization
2. VRAM vs quality trade-offs
3. single-stream decode speed
4. KV-cache quantization
5. long-context memory use

Best fit:
Single-user local inference on NVIDIA cards such as RTX-class GPUs.

Practical project:
Run one model at several EXL2 bitrates and compare VRAM usage, tokens/sec, and answer quality.

Source:
https://github.com/turboderp-org/exllamav2

### ml-explore/mlx-lm
Tags: #mlx-lm #apple-silicon #local-llm
Connected: depends on "10 GitHub repos for local LLM inference"

Focus: Apple Silicon local inference

What it is:
Apple-native LLM tooling built on MLX for M-series chips and unified memory.

Why it matters for Data Scientists:
It makes Macs unusually capable local inference machines because CPU and GPU share the same memory pool, reducing data movement overhead.

What to focus on first:
1. unified-memory intuition
2. 4-bit / 8-bit quantization
3. model loading and generation
4. memory limits by Mac configuration
5. batch vs interactive inference

Best fit:
Local experimentation on Apple Silicon where simplicity and large unified memory matter.

Practical project:
Run the same model in 4-bit and higher precision, then compare memory usage, latency, and quality.

Source:
https://github.com/ml-explore/mlx-lm

### NVIDIA/TensorRT-LLM
Tags: #tensorrt-llm #nvidia #inference
Connected: depends on "10 GitHub repos for local LLM inference"

Focus: maximum NVIDIA inference performance

What it is:
NVIDIA's dedicated toolkit for compiling and optimizing LLM inference on CUDA hardware.

Why it matters for Data Scientists:
It exposes the hardware-level performance layer behind production LLM systems, including fused kernels, low-precision execution, and in-flight batching.

What to focus on first:
1. FP8 / INT4 execution
2. kernel fusion
3. in-flight batching
4. latency vs throughput trade-offs
5. deployment complexity

Best fit:
NVIDIA-heavy environments where squeezing out maximum inference performance matters more than setup simplicity.

Practical project:
Benchmark the same model with a standard PyTorch/Transformers path and TensorRT-LLM, then compare latency and throughput.

Source:
https://github.com/NVIDIA/TensorRT-LLM

### dottxt-ai/outlines
Tags: #outlines #structured-output #json
Connected: depends on "10 GitHub repos for local LLM inference"

Focus: deterministic structured generation

What it is:
A structured text-generation library that constrains model outputs using regex, JSON schemas, and grammars.

Why it matters for Data Scientists:
Many analytical workflows need reliable machine-readable outputs. Outlines constrains generation so outputs can follow a schema instead of depending on retries and prompt wording alone.

What to focus on first:
1. JSON-schema constrained generation
2. regex constraints
3. grammar-based decoding
4. Pydantic integration
5. backend integration with local inference engines

Best fit:
Extraction, classification, synthetic-data generation, or any workflow where malformed output breaks downstream code.

Practical project:
Force a local model to return a validated Pydantic object for 1,000 rows and measure parse failures versus prompt-only JSON generation.

Source:
https://github.com/dottxt-ai/outlines

### ollama/ollama
Tags: #ollama #local-llm #prototyping
Connected: depends on "10 GitHub repos for local LLM inference"

Focus: easiest local model setup

What it is:
A local model runtime and CLI that simplifies pulling models, handling quantization, and using CPU/GPU acceleration without manual conversion or compilation.

Why it matters for Data Scientists:
It is one of the fastest ways to get from zero to a working local model for notebooks, scripts, prototyping, tool calling, and multimodal experiments.

What to focus on first:
1. model pulling and Modelfiles
2. Python SDK usage
3. structured outputs / tool calling
4. local API integration
5. model and memory selection

Best fit:
Quick experiments where setup speed matters more than low-level control.

Practical project:
Build a notebook that sends a dataframe row to a local model through Ollama and returns a structured classification plus explanation.

Source:
https://github.com/ollama/ollama

### exo-explore/exo
Tags: #exo #distributed #local-llm
Connected: depends on "10 GitHub repos for local LLM inference"

Focus: distributed local inference across devices

What it is:
A decentralized inference framework that can split models across multiple heterogeneous devices on a local network.

Why it matters for Data Scientists:
It explores a different way to overcome local memory limits: pool several consumer devices instead of buying a single larger machine.

What to focus on first:
1. model partitioning
2. device discovery
3. network bottlenecks
4. heterogeneous hardware trade-offs
5. fault tolerance / orchestration

Best fit:
Experiments with very large models that do not fit in one machine's memory.

Practical project:
Combine two local devices, run a model that is too large for one of them alone, and measure the latency cost of distribution.

Source:
https://github.com/exo-explore/exo

### flashinfer-ai/flashinfer
Tags: #flashinfer #kernels #gpu
Connected: depends on "10 GitHub repos for local LLM inference"

Focus: low-level GPU inference kernels

What it is:
A high-performance GPU kernel library for LLM serving and generation, including attention kernels used by modern inference backends.

Why it matters for Data Scientists:
You may not need to use it directly, but it helps explain where inference speed actually comes from when working below high-level serving frameworks.

What to focus on first:
1. paged attention
2. grouped-query attention
3. decode kernels
4. KV-cache interaction
5. integration with custom PyTorch loops

Best fit:
Custom inference systems, performance research, or engineers building below the framework layer.

Practical project:
Benchmark a custom attention-heavy generation loop with optimized kernels and compare decode latency.

Source:
https://github.com/flashinfer-ai/flashinfer

## Connections
- **10 GitHub repos every Data Scientist should bookmark in 2026** → **rasbt/LLMs-from-scratch** — "foundation: understand how LLMs work"
- **10 GitHub repos every Data Scientist should bookmark in 2026** → **karpathy/nn-zero-to-hero** — "foundation: rebuild deep-learning intuition"
- **10 GitHub repos every Data Scientist should bookmark in 2026** → **mlabonne/llm-course** — "model specialization and fine-tuning"
- **10 GitHub repos every Data Scientist should bookmark in 2026** → **DataTalksClub/llm-zoomcamp** — "application layer: RAG and production"
- **10 GitHub repos every Data Scientist should bookmark in 2026** → **microsoft/generative-ai-for-beginners** — "application layer: full-stack generative AI"
- **10 GitHub repos every Data Scientist should bookmark in 2026** → **Shubhamsaboo/awesome-llm-apps** — "application layer: reference architectures"
- **10 GitHub repos every Data Scientist should bookmark in 2026** → **GokuMohandas/Made-With-ML** — "production layer: MLOps and systems"
- **10 GitHub repos every Data Scientist should bookmark in 2026** → **vllm-project/vllm** — "production layer: LLM serving and inference"
- **10 GitHub repos every Data Scientist should bookmark in 2026** → **langchain-ai/langgraph** — "agent layer: orchestration and stateful workflows"
- **10 GitHub repos every Data Scientist should bookmark in 2026** → **dair-ai/Prompt-Engineering-Guide** — "agent layer: prompting, grounding, and context engineering"
- **10 GitHub repos for local LLM inference** → **ggerganov/llama.cpp + llama-cpp-python** — "CPU and hybrid local inference foundation"
- **10 GitHub repos for local LLM inference** → **vllm-project/vllm** — "high-throughput GPU inference"
- **10 GitHub repos for local LLM inference** → **sgl-project/sglang** — "efficient repeated-context and structured inference"
- **10 GitHub repos for local LLM inference** → **turboderp/exllamav2** — "fast quantized inference on consumer NVIDIA GPUs"
- **10 GitHub repos for local LLM inference** → **ml-explore/mlx-lm** — "Apple Silicon and unified-memory inference"
- **10 GitHub repos for local LLM inference** → **NVIDIA/TensorRT-LLM** — "maximum performance on NVIDIA hardware"
- **10 GitHub repos for local LLM inference** → **dottxt-ai/outlines** — "structured and deterministic output control"
- **10 GitHub repos for local LLM inference** → **ollama/ollama** — "fastest zero-config local experimentation"
- **10 GitHub repos for local LLM inference** → **exo-explore/exo** — "distributed local inference across consumer devices"
- **10 GitHub repos for local LLM inference** → **flashinfer-ai/flashinfer** — "low-level attention and decode kernels"

---
_Shared from [Mindlify](https://mindlify.co) — AI-powered thought networks_