Data Science Foundations and Exploratory Analysis Training Course

Participants can organise and run basic Python analysis workflows in Jupyter Notebook.

• Jupyter Notebook cells organised for code, narrative, outputs and reruns
• Python variables, lists, dictionaries and functions used for analysis
• NumPy arrays support vectorised calculations and missing-value-aware operations
• Files loaded with pathlib, CSV parameters and explicit encodings
• Reusable functions separate data transformation from presentation logic
• Exceptions and assertions expose unexpected inputs before downstream analysis

Participants can structure, combine and reshape tabular datasets for analysis.

• pandas DataFrames inspected through shape, dtypes, indexes and memory usage
• Tidy-data principles assign one variable, observation and value per structure
• Column selection, filtering and sorting express auditable analytical subsets
• merge, concat and join operations reconcile related operational datasets
• groupby aggregations produce customer, branch and product performance summaries
• Wide and long reshaping prepares tables for modelling and visualisation

Participants can assess data quality and document defensible cleaning decisions.

• Missingness patterns classified as MCAR, MAR or MNAR assumptions
• Duplicate records identified using business keys and matching rules
• Range, type and referential checks test source-data validity
• Outliers investigated with domain thresholds, IQR fences and z-scores
• Cleaning decisions documented in a data-quality issue log
• Train-test contamination prevented during imputation, scaling and feature construction

Participants can investigate distributions, relationships and anomalies before modelling.

• Univariate distributions profiled using frequencies, histograms and summary statistics
• Median, mean, quantiles and dispersion compared for skewed variables
• Cross-tabulations reveal segment differences in categorical outcome rates
• Correlation matrices screened for linear association and multicollinearity signals
• Scatterplots expose nonlinear patterns, clusters, censored values and anomalies
• EDA questions trace observed patterns back to operational processes

Participants can create accurate visualisations that support specific decisions.

• Chart selection matches comparison, composition, distribution, relationship and change questions
• Matplotlib and seaborn charts apply truthful axes, labels and annotations
• Small multiples compare regions, customer cohorts and reporting periods
• Colour palettes distinguish categories while remaining accessible for colour-vision differences
• Annotation techniques explain inflection points, exceptions and decision-relevant context
• Dashboard wireframes prioritise metrics, filters, drilldowns and executive actions

Participants can use statistical inference to assess evidence and uncertainty.

• Sampling distributions explain uncertainty in estimates from finite datasets
• Confidence intervals quantify plausible ranges around means, proportions and differences
• Null and alternative hypotheses frame business claims for statistical testing
• t-tests, chi-square tests and ANOVA match variable types and questions
• Effect sizes and practical significance accompany p-values in recommendations
• Multiple-comparison risks controlled through pre-specified tests and adjusted thresholds

Participants can write SQL queries that retrieve and aggregate analysis-ready data.

• Relational schemas interpreted through primary keys, foreign keys and cardinality
• SELECT, WHERE and ORDER BY retrieve defined analytical populations
• GROUP BY and HAVING calculate controlled aggregate measures
• INNER, LEFT and anti joins identify matched and unmatched records
• Common table expressions structure multi-step queries for peer review
• Window functions calculate rankings, rolling measures and customer histories

Participants can create and document features suitable for analytical models.

• Feature definitions align prediction time, source availability and decision use
• Categorical variables encoded with one-hot, ordinal and target-aware approaches
• Numerical transformations apply logarithms, binning, scaling and winsorisation appropriately
• Date fields generate recency, frequency, seasonality and elapsed-time features
• Interaction features tested against domain hypotheses and validation performance
• Feature lineage records definitions, transformations, owners and refresh dependencies

Participants can build, assess and explain regression models for continuous outcomes.

• Linear regression estimates continuous outcomes using interpretable coefficient relationships
• Residual plots assess linearity, variance, normality and influential observations
• R-squared, MAE and RMSE compare predictive accuracy appropriately
• Regularisation with Ridge and Lasso controls coefficient instability
• Regression coefficients translated into conditional effects and decision implications
• Prediction intervals communicate individual forecast uncertainty beyond point estimates

Participants can build and assess classifiers for event and risk prediction.

• Logistic regression estimates event probabilities from labelled historical records
• Decision trees expose split rules, interactions and overfitting risks
• Random forests and gradient boosting compared for tabular prediction
• Confusion matrices distinguish true positives, false positives and false negatives
• Precision, recall, specificity and F1 match asymmetric error costs
• Classification thresholds selected using capacity, risk appetite and intervention costs

Participants can design validation approaches and diagnose model performance issues.

• Holdout splits and cross-validation estimate generalisation beyond training data
• Stratified sampling preserves rare-class proportions across train and test sets
• Temporal splits prevent future information entering historical model training
• Baseline models establish minimum performance before complex model selection
• Learning curves diagnose underfitting, overfitting and data sufficiency
• Error analysis segments failures by customer, product, geography and time

Participants can develop descriptive segments and assess their operational usefulness.

• Distance measures and scaling determine meaningful similarity between observations
• k-means clustering partitions records using centroid-based iterative optimisation
• Elbow plots and silhouette scores assess candidate cluster solutions
• Cluster profiles combine behaviours, value measures and operational attributes
• Principal component analysis reduces correlated variables while retaining variance structure
• Segmentation recommendations distinguish descriptive groups from actionable treatment policies

Participants can produce and evaluate forecasts from time-indexed data.

• Time-indexed data checked for gaps, duplicates, frequency and calendar effects
• Trend, seasonality and residual components decomposed using STL
• Lagged features represent persistence, seasonality and known future drivers
• Rolling-origin validation simulates sequential forecasting decisions without future leakage
• Naive, seasonal-naive and moving-average forecasts establish practical benchmarks
• Forecast error measured with MAE, RMSE, MAPE and bias

Participants can convert unstructured text into usable analytical features and findings.

• Text fields cleaned through tokenisation, normalisation, stopword removal and stemming
• Document-term matrices represent word occurrence for quantitative analysis
• TF-IDF weighting identifies terms distinctive within documents and collections
• Sentiment lexicons classify expressed polarity while recording contextual limitations
• Topic modelling with LDA discovers recurring themes in document sets
• Text classification evaluation checks class balance, errors and harmful patterns

Participants can design experiments and distinguish association from causal evidence.

• Randomised controlled trials separate treatment effects from background variation
• Assignment units, eligibility rules and exposure windows prevent contamination
• Power calculations relate sample size, minimum effect and decision risk
• A/B test metrics define primary outcomes, guardrails and stopping rules
• Difference-in-differences estimates policy effects using comparison groups and periods
• Experiment readouts report uncertainty, practical impact and implementation recommendations

Participants can identify governance, fairness and human-review requirements for analytical work.

• Personal data classified by sensitivity, purpose, retention and access requirements
• Data minimisation limits collection to stated analytical purposes
• Fairness assessments compare error rates across protected and affected groups
• Proxy variables examined for indirect discrimination and unfair exclusion
• Model cards document intended use, limitations, performance and review ownership
• Human review triggers defined for high-impact automated recommendations

Participants can structure data science work for review, reruns and ongoing monitoring.

• Project folders separate raw data, processed data, code and outputs
• Git commits record meaningful analytical changes and peer-review history
• Requirements files specify reproducible package versions and runtime dependencies
• Random seeds and configuration files support repeatable model results
• Data pipelines separate ingestion, transformation, modelling and reporting stages
• Monitoring plans track drift, performance decay, incidents and retraining triggers

Participants can communicate evidence, uncertainty and recommended actions to decision-makers.

• Stakeholder questions translated into decision statements, metrics and analytical boundaries
• Executive summaries state finding, evidence, uncertainty and recommended action
• Data stories combine annotated visuals, business context and decision consequences
• Assumptions, limitations and alternative explanations disclosed without technical obscurity
• Model performance presented against baseline, cost and operational capacity
• Action plans assign owners, deadlines, controls and follow-up measures

Participants can build, review and present a complete data science project with documented analysis, validated results and implementation recommendations.

• Frame a decision question using stakeholder requirements and measurable success criteria
• Acquire, join and document source data with reproducible Python workflows
• Profile quality, missingness, distributions and relationships using targeted exploratory analysis
• Build and validate an appropriate predictive, descriptive or forecasting model
• Produce decision-focused visualisations, findings, limitations and recommended operational actions
• Present a model card, reproducibility record and implementation monitoring plan