Unit 8 Project: The AI Continuous Improvement Plan
Unit 8 Final Project – The AI Continuous Improvement Plan | Knowledge Pillars AIASSkip to main content
Unit 8 · Final Project
The AI Continuous Improvement Plan
This is the capstone project for Unit 8 — and for the AIAS programme. You will apply every lesson from the unit to a real or realistic AI system: scanning for trends, evaluating performance, designing an experiment, proposing an innovation, and reflecting on the process. This is your opportunity to demonstrate that continuous improvement is not a set of techniques but a professional mindset.
The AI Continuous Improvement Plan is a structured professional document that any AI specialist or strategist might produce in a real workplace. It demonstrates not just what you know, but that you can apply the full cycle: scanning the environment for what's possible, measuring current performance rigorously, designing a disciplined experiment, recommending a strategic innovation, and reflecting on the process honestly.
The plan has five parts — one per Unit 8 lesson. Each part is deliberately concise: this is a working document, not an essay. Precision and specificity matter more than length.
The most important instruction in this project brief
Every part of this plan must be grounded in a specific AI system — not a hypothetical or abstract one. Choose a real or realistic system you know well enough to write about concretely. Generic plans ("the company should use AI to improve things") score in the lowest rubric band. Specific plans ("the churn prediction model's recall for high-value customers is 62%, below our 80% target") demonstrate actual capability.
Vocabulary requirement
Across all five parts, you must correctly use at least 10 key terms from Unit 8's vocabulary (drawn from lessons 8.1–8.5). Terms used correctly and in context demonstrate mastery; terms inserted artificially without meaningful use do not count. The AI Project Coach can help you identify where terms apply naturally to your content.
Step 1 — Choose Your Context
Select your AI system scenario
Choose a scenario below to work with throughout all five parts. If you have a real workplace AI system, use that — it will produce the most authentic and useful plan. If not, choose one of the provided scenarios, which have enough detail to write specifically about.
Scenario Selector
Select one scenario · All five project parts will use this context
Step 2 — Complete All Five Parts
Build your Continuous Improvement Plan
Work through each part in sequence. Use the requirements, exemplars, and vocabulary hints as guides. The AI Project Coach (below) is available at any point if you need help applying a concept to your chosen scenario.
🔭
Part 1 — Environmental Scan (Lesson 8.1)
Identify what's emerging that could change what's possible for your system
Requirements
1
Identify 2–3 specific emerging AI developments relevant to your chosen system. These must be real developments (published research, commercial tools, or industry-documented trends) — not generic AI ideas.
2
For each development, explain why it's specifically relevant to your system's domain, users, or limitations — not just why it's interesting generally.
3
Name at least one concrete source where you would monitor this development (a publication, repository, conference, or industry report).
4
Indicate where each development sits on the Tech Adoption Lifecycle for your industry and what that timing means for your organisation.
Exemplar (RetailCo context)
"1. Session-based recommendation models (e.g., transformer-based architectures like SASRec): currently in the Early Adopter phase in e-commerce. Unlike collaborative filtering, these use the sequence of a user's current session — not historical purchase data — to predict intent. For RetailCo this is directly relevant because our recommendation model fails for new and infrequent users who have no purchase history. Source to monitor: arXiv cs.IR (Information Retrieval) and Hugging Face model releases. 2. Real-time personalisation via streaming data: several major retailers (Amazon, Zalando) now update recommendation models with each click rather than overnight batch retraining. RetailCo's model is batch-retrained weekly, creating a 7-day lag before behavioural signals affect recommendations…"
Vocabulary to consider using: Emerging Technology, Tech Adoption Lifecycle, Early Adopter, Sustaining vs. Disruptive Innovation, model drift, incremental delivery.
0 charactersAim for 250–400 words
📊
Part 2 — Performance Evaluation (Lesson 8.2)
Diagnose the current state with specificity — using the right metrics for the right reasons
Requirements
1
State the system's current performance on 2–3 specific metrics with actual or estimated values. Explain why these metrics (and not others) are the right ones to evaluate this system.
2
Identify one metric that is underperforming relative to a benchmark, target, or competitor. Quantify the gap — don't just say "it's low."
3
Conduct a brief error analysis: describe what pattern of failures or underperformance you observe. What type of cases or user segments is the system getting wrong? What does this reveal about where the next improvement should focus?
4
Explain what evaluation method you would use to measure improvement: hold-out validation, A/B test, cross-validation, or production monitoring — and why that method is appropriate for this system.
Exemplar (MedCenter context)
"Current metrics: Recall: 68% (target: 80%); Precision: 74%; F1-score: 0.71. Recall is the primary metric because in a readmission risk context, a false negative (missing a high-risk patient) has far greater cost than a false positive — an unnecessary follow-up call is cheap; an avoidable readmission costs $15,000+. The gap: our recall is 12 points below target, meaning roughly 1 in 3 high-risk patients is not flagged. Error analysis: the model systematically underperforms on patients discharged after short stays (under 3 days) and on patients over 80 — both groups that were underrepresented in the original training data from 18 months ago. Evaluation method: I would use a time-based hold-out validation…"
Vocabulary to consider using: Precision, Recall, F1-score, RMSE, confusion matrix, A/B testing, cross-validation, model drift, performance metric, error analysis.
0 charactersAim for 250–400 words
🔬
Part 3 — PDCA Experiment Design (Lesson 8.3)
Design one complete, disciplined improvement experiment grounded in your Part 2 findings
Requirements
P
Plan: Write your hypothesis in this exact form: "We believe [specific change] will [measurable outcome] because [evidence-based reasoning], measurable by [metric] on [validation method]." Then state your pre-defined success threshold.
D
Do: Describe exactly what will be implemented, at what scale, and with what controls. What will be kept constant? What is the smallest valid implementation of this experiment?
C
Check: Define in advance what a confirmed result looks like vs. a disconfirmed result. What unexpected outcomes would you look for? How will you distinguish real improvement from noise?
A
Act: Describe both paths — what happens if the hypothesis is confirmed (how do you scale, what is the next hypothesis?), and what happens if it isn't (what does the null result reveal, and what does that inform about the next experiment?).
Exemplar (FinanceFirst context)
"Plan: We believe adding applicant debt-to-income ratio trend over 6 months (not just current ratio) will improve precision by at least 8 points (from 58% to 66%) without reducing recall below 80%, because our error analysis shows we misclassify borrowers whose income recently declined — a pattern the trend variable would capture. Measurable by held-out test set (most recent 6 months of applications, withheld from training). Do: Engineer the trend feature from existing credit bureau data (no new data source required). Retrain the model with this feature added; all other features and architecture unchanged. Evaluate on the same held-out test set used for the baseline. Check: Confirmed if precision ≥ 66% with recall ≥ 80%..."
Propose two innovations — one sustaining, one disruptive — using the five-step proposal framework
Requirements — Two proposals, each using the five-step framework
S
Sustaining innovation proposal (5 steps): Challenge → Solution → Two-sided case (benefit of acting + cost of not acting) → Pilot design → Roadmap. This should be a near-term, lower-risk improvement that makes your current system meaningfully better.
D
Disruptive innovation proposal (5 steps, can be briefer): A longer-horizon innovation that would create a new capability, market, or service that does not currently exist for your organisation. Must explain what makes this hard for competitors to replicate.
★
For at least one proposal, explicitly identify what your organisation uniquely has (proprietary data, domain expertise, customer relationships) that makes this innovation more defensible than a generic capability any competitor could buy.
Exemplar (ManufacturePro context)
"Sustaining: Real-time IoT integration. Challenge: our model uses only historical maintenance logs — there is a 7-day lag before new sensor patterns affect predictions, causing missed early warning signals. Solution: ingest real-time vibration and temperature readings from our 340 installed sensors into the model's feature pipeline. Two-sided case: integrating real-time data is projected to reduce missed failure predictions by 30% (estimated $54K/month reduction in unplanned downtime); not acting means competitors adopting edge AI will achieve operational efficiency we cannot match. Pilot: connect 40 sensors from Line 3 (lowest production priority) for 60 days; compare downtime on Line 3 vs. historical baseline… Disruptive: AI-powered maintenance-as-a-service for SMEs. Use our 8 years of proprietary failure-pattern data…"
Vocabulary to consider using: Competitive Advantage, Sustaining vs. Disruptive Innovation, Pilot Program, Innovation Roadmap, Sustainable Advantage, Emerging Technology, two-sided case, Tech Adoption Lifecycle, ROI.
0 charactersAim for 350–500 words
🪞
Part 5 — Sprint Retrospective (Lesson 8.5)
Reflect on the process of creating this plan — not just what you produced, but what you learned doing it
Requirements — Address all four retrospective questions
1
What went well? Which part of the plan came most naturally? What specific practice, approach, or thinking process produced your strongest work — and how will you consciously carry it into future projects?
2
What was most difficult? Where did you struggle with specificity, evidence, or applying a concept to your chosen system? Be honest — vague reflection scores the same as no reflection.
3
What did this process reveal? Did creating the plan surface something about your chosen AI system that you hadn't previously articulated? Did the act of writing the PDCA experiment clarify what the actual bottleneck is? Did the innovation proposal reveal a competitive gap you hadn't fully considered?
4
What is your personal next iteration? If this plan is your first sprint, what would sprint 2 of your own professional development look like? Name one specific skill, concept, or practice from Unit 8 you want to develop further — and a concrete first action you will take to do so.
Exemplar response (any context)
"What went well: the PDCA experiment design (Part 3) came most naturally — I found that having to write the hypothesis in the prescribed format ("We believe… because… measurable by…") forced me to be specific in a way my previous experiment planning hadn't been. I'll use that format as a default going forward. What was most difficult: the two-sided innovation case (Part 4). I struggled to quantify the cost of inaction beyond vague competitive language. I need to be better at translating strategic threats into projected financial or market-share terms. What was revealed: writing Part 2 forced me to actually look up our model's precision and recall together — I'd only been tracking accuracy. I discovered our precision is significantly lower than I'd assumed, which completely changed which improvement to prioritise in Part 3. Personal next iteration: I will propose adding one experimentation-focused Key Result to our Q3 OKRs — "complete 2 PDCA experiments on the recommendation model." First action: draft the proposed OKR and share it with my manager this week."
Select the level that honestly describes your performance in each criterion. The rubric reflects the official Unit 8 assessment criteria. Honest self-assessment — including identifying where you fell short — demonstrates the iterative mindset better than selecting all A's.
Unit 8 Self-Assessment Rubric
Select one level per criterion · Then submit for your overall self-assessment
Criteria 1: Active Participation & Engagement Did you engage fully with Unit 8's materials, discussions, and activities?
Criteria 2: Application of Innovation & Continuous Improvement Strategies How effectively did you apply agile iteration, metrics, and innovation proposals in your plan?
Criteria 3: Use of Unit Vocabulary & Concepts Did you correctly and meaningfully use key terms from Unit 8 across your plan?
Criteria 4: Iterative & Reflective Thinking Does your retrospective demonstrate genuine continuous improvement mindset?
Self-assessed grades: Select all four criteria above
AI Project Coach — Available Throughout
Get help with any part of your plan
The AI Project Coach is available throughout the project — before you start a part, while you're writing, or after you've drafted something and want feedback. Ask about how to apply a specific concept to your scenario, how to make your hypothesis more specific, what your error analysis reveals, or how to strengthen your innovation case.
KP
AI Project Coach — Knowledge Pillars AIAS
Helping you apply Unit 8 concepts to your specific scenario
KP
Hi! I'm your Project Coach for the AI Continuous Improvement Plan. Ask me how to sharpen your hypothesis, what the right metric is for your system, how to frame your innovation's two-sided case, what your error analysis reveals about your next experiment, or how to write a retrospective that's genuinely specific rather than generic. What are you working on?
🏆✨🎓
AIAS Programme Complete!
You have completed the AI Implementation and Advisory Specialist programme — all eight units, from understanding AI technologies and trends through to continuous improvement and innovation. You have built a professional capability that spans the full lifecycle of AI: from foundations to deployment, from client communication to strategic recommendation, and from measurement to the iterative mindset that keeps AI systems improving long after they launch.
This final project is not the end of the process — it is the first sprint of your professional practice.
