MATHEMATICS & DATA / SPECIALIST

DAT220Large-scale data processing & Spark

24 h productive allocation9 h 36 m learning + 14 h 24 m training4 units · 1 assessed practical brief

Detailed course plan and training brief. A complete taught course pack is in preparation. The linked foundation module, where shown, is a smaller available resource.

BEFORE YOU BEGIN

A clear starting point.

Prerequisites: DAT210

Specialist studio; environment and faculty must be confirmed before scheduling

WHAT YOU SHOULD BE ABLE TO DEMONSTRATE

Explain the assumptions and failure modes in distributed tables.
Complete the practical: run a small local Spark transformation and explain the execution plan and why it would or would not scale.
Defend the resulting transform script with reproducible evidence.

THE SUBJECTS / FOUR CONNECTED UNITS

What you will study.

UNIT 016 h allocated

Distributed tables

Partitioning
lazy evaluation
execution plans
shuffles

UNIT 026 h allocated

Transformations

Joins
aggregations
window functions
skew

UNIT 036 h allocated

ML data preparation

Feature tables
time-aware joins
reproducibility
resource budgets

UNIT 046 h allocated

Performance and quality

Caching
partition tuning
reconciliation
failure diagnosis

THE TRAINING STUDIO

Make the subject
visible in your work.

Run a small local Spark transformation and explain the execution plan and why it would or would not scale.

Evidence to produce

Transform script
execution-plan note
reconciliation report
resource profile

Training brief for supervised delivery. Code, datasets, answer keys and environment acceptance must be completed in the full course pack before delivery is confirmed.

ASSESS THE REASONING

A question you
should be able to answer.

Why is distributed processing not automatically faster for a small dataset?

Review dimensions

Technical correctness · reproducibility · interpretation of results · failure analysis · individual explanation. Course-level rubric and faculty moderation form part of the final adopted teaching pack.

Continue in the official reference ↗Programme assessment framework →

External reference access, hardware, cloud accounts and paid services are separate. No third-party certification is included by implication.