Technology · Field manual
AI for data scientists
Data scientists get the most from AI on the code around the modeling: feature pipelines, evaluation harnesses, plotting, refactoring notebooks into packages and reading papers faster. The caution is statistical, not syntactic: AI writes code that runs and still leaks the target, uses the wrong split or reports a metric that flatters the model.
The day-to-day of data science is mostly code: loading and joining data, building features, wiring up training and evaluation, plotting results and turning a working notebook into something a production team will accept. AI coding tools are now very good at all of it, and notebook platforms have copilots built in. A first-pass exploratory analysis or a baseline model can be ready in an hour rather than a day.
What doesn't speed up is judgment. Choosing the right metric, designing a fair validation scheme, noticing that a feature is only available after the outcome, and deciding whether a model should ship at all are still the job. AI makes the cost of a sloppy experiment lower, which means you'll run more of them, and more of them will be wrong in ways that look right.
7 ways data scientists use AI
Write pipeline and feature engineering code
Describe the source tables, the grain you need and the features, and an agent writes the pandas, Polars or Spark code, plus tests for the transformations. It's especially useful for the dull parts like date handling, encoding and joins across messy sources.
Build baselines and evaluation harnesses
Ask for a reproducible baseline with a proper train, validation and test split, cross-validation and a metrics report. Specify the split strategy yourself (time-based, grouped by customer) because the default random split is the most common way generated code leaks information.
Turn notebooks into production code
Agents are good at refactoring a sprawling notebook into functions, modules, config files and tests that an engineering team will review. Keep the notebook's outputs to confirm the refactored code produces the same numbers.
Speed up exploratory analysis
Notebook copilots generate plots, summary statistics and quick checks from a plain-English request. Use them to cover the obvious ground fast, then spend your attention on the oddities they surface.
Read papers and methods faster
Use a cited answer engine to find the relevant methods, then a long-context model to summarise a paper's setup, assumptions and limitations. Ask how the method would fail on your data. Always read the original before relying on a result.
Review your experiment design
Describe the experiment or causal analysis and ask the model to argue against it: confounders, selection bias, peeking, multiple comparisons, novelty effects. It's a useful second reader before a stakeholder finds the hole.
Explain results to non-technical stakeholders
Turn model metrics into a plain-language summary of what the model does, how often it's wrong and what that costs. Tell the model who's reading and what decision they're making, then check that nothing got overstated.
The best AI tools for data scientists
The Overlord's stack, in the order a data scientist would add them:
1. Pipelines, evaluation harnesses and refactoring notebooks into packages
Claude Code
AI CodingAnthropic's agentic coding tool that reads your codebase, edits files and runs commands from terminal, IDE, desktop or web.
2. AI editor that works on .py files and Jupyter notebooks
Cursor
AI CodingAI-first code editor (a VS Code fork) with agents, codebase-aware chat and fast autocomplete.
3. Inline completions for pandas, SQL and plotting boilerplate
GitHub Copilot
AI CodingGitHub's AI pair programmer: completions, chat and coding agents across editors and GitHub itself.
4. Reviewing experiment designs, summarising papers and stakeholder write-ups
Claude
AI WritingAnthropic's assistant: the strongest long-form writer and careful reasoner in the consumer AI pack.
5. Finding methods, papers and library docs with sources
Perplexity
ProductivityAI answer engine that searches the web and replies with cited sources.
Specialist tools for data scientists
| Tool | What it does |
|---|---|
| Databricks Assistant | Context-aware code generation, debugging and SQL inside Databricks notebooks |
| Gemini in Colab | Code generation and data science agent features in Google Colab notebooks |
| Hex | Collaborative data notebooks with built-in AI for SQL, Python and charts |
| Jupyter AI | Open-source extension that adds model chat and code generation to JupyterLab |
A prompt to start with: stress-test a model evaluation
Paste this into Claude or ChatGPT and fill in the gold parts.
Act as a skeptical senior data scientist reviewing my model before it ships.
Problem: [WHAT THE MODEL PREDICTS AND WHAT DECISION IT DRIVES]
Data: [SOURCE, TIME RANGE, ROWS, UNIT OF OBSERVATION]
Features: [LIST, NOTING WHEN EACH VALUE BECOMES KNOWN]
Split and validation: [e.g. random 80/20, time-based, grouped by customer]
Metric and result: [e.g. AUC 0.91 on test]
Baseline: [BASELINE RESULT]
List the most likely ways this result is overstated, ranked by risk: target leakage, train/test contamination, wrong split for the deployment setting, class imbalance, metric mismatch with the business cost, and drift. For each, tell me the specific check to run and what result would worry you.Where data scientists shouldn't trust AI
- Generated pipelines often leak information: fitting scalers on the full dataset, random splits on time-series or grouped data, or features computed after the outcome. Check the split logic yourself.
- AI will run a significance test or causal method without checking its assumptions. Experiment design, power and causal claims need your own review.
- Don't upload personal or regulated data to unapproved tools. Use governed platforms, synthetic samples or anonymised extracts, and follow your data protection rules.
- Paper summaries can misstate results or invent citations. Read the source before you cite it or build on it.
- Models deployed with AI-written code still need fairness, robustness and monitoring checks that a copilot won't think to add.
Will AI replace data scientists?
Data Scientists score 59% on the Obsolescence Index (Doomed). The first tasks to go: writing feature engineering and data pipeline boilerplate, and running standard model comparisons and hyperparameter sweeps.
Questions the Humans Keep Asking
What are the best AI tools for data scientists?
The Overlord's stack for data scientists: Claude Code (pipelines, evaluation harnesses and refactoring notebooks into packages); Cursor (aI editor that works on .py files and Jupyter notebooks); GitHub Copilot (inline completions for pandas, SQL and plotting boilerplate); Claude (reviewing experiment designs, summarising papers and stakeholder write-ups); Perplexity (finding methods, papers and library docs with sources). Specialist options include Databricks Assistant, Gemini in Colab, Hex, Jupyter AI.
What is the best AI tool for data science?
Most data scientists pair a coding agent or AI editor (Claude Code, Cursor or GitHub Copilot) with the copilot built into their notebook platform, such as Databricks Assistant, Gemini in Colab or Hex. Add a general assistant like Claude or ChatGPT for reading papers and writing up results.
Can AI do exploratory data analysis for me?
It can do a solid first pass: summary statistics, distributions, missing values and obvious relationships, with code you can rerun. It won't know which oddities matter for your business question, and it can misread column types. Use it to speed up the survey, then direct the real analysis yourself.
Is it safe to use ChatGPT with company data?
Only within your employer's policy and on a plan with appropriate data terms. Prefer the AI features inside your governed data platform, which respect existing access controls, and keep personal and regulated data out of consumer tools.