Book Notes: The Mythical Man Month: Essays on Software Engineering

Overview

This post contains my notes on the book Team Topologies: Organizing Business and Technology Teams for Fast Flow by Matthew Skelton, Manuel Pais, and Ruth Malan.

You can find the book on Amazon

I’ll be adding my notes to this post as I read through the book. The notes will be organized by chapter and will include key concepts, code examples, and any additional insights I find useful.

Chapter 1: The Tar Pit

Programming Systems Product

• Programming Product costs ~3x as much as a debugged program w/ same function

The Joys of the Craft

1. Making Things
2. Making things that are useful
3. Fashioning complex puzzle-like objects
4. Always learning
5. Delight of medium

The Woes of the Craft

1. Perfection
2. Other people’s objectives
3. Other people’s systems
4. finding bugs is work
5. product appears obsolete before finished

Challenge and Mission: to find real solutions to real problems on actual schedules with available resources.

Chapter 2: The Mythical Man-Month

Calendar Time Causes Issues

1. Estimating is poorly developed
2. Estimating techniques confuses effort with progress
3. Estimates are uncertain
4. Schedule progress is poorly monitored
5. Natural Reaction to Schedule Slippage is to add manpower

Optimism: All programmers are optimists

Creative activities:

1. idea
2. implementation
3. interaction

Incompleteness and Inconsistencies become clear during implementation.

The man-month

Cost = # of programmers * number of months

The bearing of a child takes 9 months, no matter how many women are assigned.

Project Planning Rules:

• 1/3 Planning
• 1/6 Coding
• 1/4 component/system test
• 1/4 system test

Gutless Estimating

• Develop and Publicize productivity Figures
• bug-incidence figures
• estimating rules

Brook’s Law: Adding manpower to a late software project makes it later.

Chapter 3: The Surgical Team

Team Composition

• 1 Surgeon: chief programmer, system architect
• 1 Co-Pilot: backup programmer, assistant architect
• 1 Administrator: handles clerical work
• 1 Editor: proofreads documentation
• 2 Secretaries: handle typing
• 1 program clerk: maintains program library
• 1 Toolsmith: builds and maintains programming tools
• 1 Tester: designs and runs tests
• 1 Language Lawyer: expert in programming language

1. Surgeon and Co-Pilot do all design and coding
2. Surgeon makes unilateral decisions

Chapter 4: Aristocracy, Democracy, and System Design

Conceptual integrity: system has a single, consistent design approach

• Most important consideration in system design
• Purpose: Ease of use

Ratio of function to conceptual complexity: test of system design

Aristocracy and Democracy

• Conceptual Integrity -> design from one mind, or from a very small number of agreeing resonant minds
• Scheduling Pressures -> Many hands

1. division of labor between architecture and implementation
2. structure programming implementation teams (surgical teams)

Separation of architectural effort from implementation effort is essential to conceptual integrity for large projects.

• Architecture: Complete and detailed specification of the user interface
• Architect: user’s agent, professional/technical knowledge
• Architecture: What
• Implementation: How

Good features/ideas that do not integrate with the system should be dropped.

Chapter 5: The Second-System Effect

Architect: works against a budget Iteration: bids->estimation upward, design downward

High Estimate:

• cut the design or
• challenge the estimate by suggesting cheaper implementations

things to remember:

• builder has inventive/creative responsibility -> suggest, not dictate
• suggest a way of implementing anything specified, accept any other way to meet objectives
• deal quietly and privately w/ suggestions
• forgo credit for ideas suggested

Self-Discipline

First Work -> Spare and Clean Second system -> tendency to over design

Chapter 6: Passing the Word

Written Specifications

Manual: External Specification

• describes and prescribes every detail

Formal Definitions

• precise definitions

A programmed simulator can be used as a formal definition

Conferences and Courts

Meetings are necessary

• Weekly half-day conference of all architects
  • propose problems/changes distributed in writing
  • discuss and decide
  • chief architect decides if no consensus
  • minutes kept and distributed
  • appeals possible
• Annual court session
  • two week long meeting
  • backlog of minor appeals, open issues, disgruntlements

Benefits:

• same group: no time needed to bring people up to date
• well versed in project: better decisions
• deeply involved in outcome
• problems -> solution within boundaries
• formality of proposals focuses attention and forces decision
• vesting of decision in chief architect avoids compromise and delay

Telephone Log

• Record each question and answer
• Logs of architects are concatenated, produced, and distributed weekly

Product Test

• independent technical auditing group
• customer is independent auditing group

Chapter 7: Why Did the Tower of Babel Fail?

A successful project needs:

• Clear Mission
• Manpower
• Materials
• Time
• Technology
• Communication
• Organization

Options for communication:

1. Informal: Telephone service, clear definition of intergroup dependencies
2. Meetings: regular technical briefings
3. Workbook: written records of decisions

Project Workbook:

• What: structure of documents
• Why:
  1. early design of structure ensures structure is crafted
  2. control of distribution of information
• Mechanics:
  1. Everyone should have access
  2. Timely updates
  3. Continual Maintenance: diff of change, summary of changes

Organization:

• Purpose: reduce communication/coordination overhead
• Principle: No man can serve two masters
• Organization: division of labor + specialization of function
• Organization subtree need:
  1. mission
  2. producer
  3. technical director/architect
  4. division of labor
  5. interface definitions among parts
• Role of Producer:
  • assembles team, divides work, establish schedule, acquires resources
  • Establishes communication outside team, upwards and sideways
  • Ensures schedule met, shifting resources and organization
• Role of Technical Director/Architect:
  • conceives design to be built, identifies sub-parts, specifies how it will look
  • unity and conceptual integrity
  • invents solutions and shifts design to solve technical problems
• Producer could be the same as Technical Director/Architect
  • strong management and strong technical talen is rarely found in one person

Rare: Thinkers Doers: Rarer Thinker-Doers: Rarest

• Producer might be boss w/ director as right hand
• Director might be boss w/ producer as right hand

Chapter 8: Calling the Shot

Estimating

What not to do:

• estimate coding portion and apply ratios from earlier
• errors in the estimate or ratios could lead to large errors in schedule

small program estimation != large program estimation

Estimating misses often due to missed:

• short unrelated jobs
• meetings
• paperwork
• company business
• sickness
• personal time

Productivity = thought per statement + errors it includes Productive time = x5 when high-level language used

Chapter 9: Ten Pounds in a Five-Pound Sack

Program Space as Cost

Size is a large part of the user cost of a programming system product

Size itself isn’t bad, but unnecessary size is.

Size Control

Project Manager: size control = technical and managerial

1. budget for all aspects of size set total size budgets as well as space budgets for major components

2. Define exactly what a module must do when you specify how big it must be.

3. Total-system, user-oriented attitude may be the most important function of the programming manager

Space Techniques

Space Budgeting + Control != small program small programs require invention and craftsmanship

Space time tradeoffs

• ensure team is trained in programming technique
• recognize programming has a technology and components needs to be fabricated

Representation is the essence of programming

rethinking the overall strategy (especially how data is represented) instead of code tricks can lead to innovation

10: The Documentary Hypothesis

Documents for a Computer Product

Critical Documents

• Objectives: defines need to be met and goals, constraints, and priorities
• Specifications: defines what the product must do to meet objectives
• Schedule
• Budget: existance forces technical decisions that would be avoided otherwise
• Organization Chart
• Space allocations
• Estimate, forcast, prices

Documents for a university department

• Objectives
• Course Descriptions
• Degree requirements
• Research proposals
• Class Schedule and teaching assignments
• Budget
• Space allocation
• Assignment of staff and graduate students

Documents for a Software Project

• Objectives
• Product Specifications
• Schedule
• Budget
• Space allocation
• Organization chart

Why have formal documents

1. writing decisions down
2. documents communicate decisions
3. documents provide database and checklist for management
4. A written plan is precise and communicable

Chapter 11: Plan to Throw One Away

Use a pilot project and throw it away to learn about the problem and solution space.

The only constancy is change itself

As projects progress, actual need and perception of need change.

Plan the system for change

Use:

• modular design
• subroutines
• definition of intermodule interfaces
• documentation
• high-level language
• self-documenting techniques
• numbered versions

Plan the organization for change

plans as tentative:

• plans
• milestones
• schedules

Keep 2-3 developers unassigned to specific tasks to handle unexpected changes.

Reassignment from technical to managerial should be “reassignment”.

Surgical Type programming teams helps solve management problems of change..

Two steps forward, one step back

Changes after delivery are “program maintenance”

• repair design defects
• maintenance costs ~40% more

Fixing a defect has 20-50% chance of introducing another defect.

• regression testing is essential, but costly

One step forward, one step back

Repairs

• destroy structure
• increase entropy
• increase disorder
• less focus on fixing design flaws
• more focus on fixing bugs introduced by fixes

Systems building: entropy-decreasing process

Program Maintenance: entropy-increasing process

Chapter 12: Sharp Tools

1. individualized tools hamper communication
2. tool lifetime: technology changes when machine or working environment changes
3. more efficient: common development and maintenance tools

Manager needs to set aside time and resources for tool building and maintenance.

What tools?

• computer facility
• operating system
• language
• utilities
• debugging aids
• test-case generators
• text-processing system

Target Machines

Target Machine: machine for which program is written Vehicle Machine: machine on which program is developed

Target Facility: target machines + operators and system programmer

• The book mentions some target specs that are likely outdated now.

Debugging machine: vehicle machine + debugging aids

Vehicle Machines and Data Services

Simulators: vehicle machine simulates target machine that’s dependable but not necessarily accurate Compiler and assembler: compile code for the target system Programming library and accounts:

• group/programmer had area where copies of code is kept
• when ready, copy passed to manager for integration
• only manager could make changes Programming Tools: Toolsmith builds and maintains programming tools Documentation System: saves the most labor
• Too much documentation: provide a map to work through important docs or work to reduce bulk
• Too little documentation: is a problem Performance Simulator: required

High-Level Language and Interactive Programming

Most important tools:

1. High-level language
2. Interactive programming system

High-level language

Pros:

• improves productivity
• debugging speed Cons:
• not flexible enough
• too big
• too slow

Interactive Programming

Pros:

• useful
• improves debugging

Chapter 13: The Whole and the Parts

Designing the Bugs Out

Conceptual integrity addresses problems of mismatched assumptions among parts.

Design the Product

• function definition
• careful specification
• exorcism of frills of function

Testing the Specification

• review by users

Top-down Design

• Design as sequence of refinements
• architecture -> implementation -> realization
• rough tasks -> rough solution, repeat until detailed design
• avoids bugs:
  • clarity of struction/representation
  • partitioning and independence
  • suppression of detail
  • tested at refinement

Structured Programming

• design programs whose structures consist only of loops and conditionals

Component Debugging

1. On-machine debugging
2. Memory Dumps
3. Snapshots
4. Interactive Debugging
5. Test Cases

System Debugging

1. Use debugged components
2. Build plenty of scaffolding
3. Control Changes
4. Add one component at a time
5. Quantize updates

Taxonomy

References:

Book Notes: Team Topologies: Organizing Business and Technology Teams for Fast Flow

Overview

This post contains my notes on the book Team Topologies: Organizing Business and Technology Teams for Fast Flow by Matthew Skelton, Manuel Pais, and Ruth Malan.

You can find the book on Amazon

I’ll be adding my notes to this post as I read through the book. The notes will be organized by chapter and will include key concepts, code examples, and any additional insights I find useful.

Chapter 1: The problem with org charts

Overview:

• Conway’s law: Software architecture and team structure are similar forces
• Team Topologies
  • clarifies team purpose and responsibilities
  • Humanistic approach to building software systems

There are 4 fundamental team types:

1. Stream-aligned team
2. Enabling team
3. Complicated subsystem team
4. Platform team

Team structures must match the desired software architecture.

Cognitive Load: The total amount of mental effort being used in the working memory.

When cognitive load exceeds a team’s capacity, it leads to burnout and reduced productivity.

Obstacles to fast flow:

• blockers
• disengaged teams
• pushing against conways law
• software too big for teams
• confusing org design
• team pulled in too many directions
• Painful re-orgs

Chapter 2: Conways Law and why it matters

Overview:

• Conway’s law: organizations design systems that mirror their communication structure
• organization constraints impact software architecture
• everyone communicating with everyone is not scaleable
• choose team structures that align with desired software architecture
• limiting communication paths improves flow

Team assignments are the first draft of the architecture.

For a safe rapid flow of change, team structures must align with the desired software architecture.

Not all communication and collaboration is good.

Repeated re-orgs damage team morale and productivity, create fiefdoms, and lead to loss of institutional knowledge.

Conways law is a fundamental principle that must be considered when designing both software systems and organizational structures.

Chapter 3: Team First Thinking

Overview:

• Teams are more effective than individuals for software delivery
• Multi-team groomings should follow Dunbars number
• Max cogntive load for a team should be less than the total cognitive load of its members
• Establish clear team boundaries and responsibilities
• For success, change the team working environment

Small long-lived teams are more effective than large short-lived teams. Smaller size creates better trust Needs of the team:

• Arrive for stand-ups on time
• Keep discussions and investigations on track
• encourage a focus on team goals
• unblock other team members
• mentor new or less experienced team members
• avoid “winning” arguments - explore options together

Limit the number and type of domains per team

• assign each domain to a single team
• single team should be able to accommodate 2-3 simple domains
• team responsible for a complex domain should not have any more domains assigned to them
• avoid a single team responsible for two complicated domains

Match software boundary size to team cognitive load Design Team APIs Facilitate team interactions to build trust Engineering Practices are foundational

Chapter 4: Static Team Topologies

Overview:

• constant change in team structures is detrimental
• critical considerations for team design
  • Technical/Cultural maturity
  • Org scale
  • Engineering discipline
• feature team/product team works in a supportive environment

Anti-patterns:

• Ad Hoc Teams
• Shuffling team members

Design For Flow Change

• low friction software delivery
• Spotify Model
  • squads
  • tribes are collection of squads
  • testers across squads are a chapter

Shape team intercommunication to enable flow and sensing

• organization optimized for flow of change avoids transition handoffs

DevOps and the DevOps Topologies

• DevOps Topologies Catalog

Successful Team Patterns

• Feature Teams require high engineering maturity and trust
• Product teams Need a support system
• Cloud teams don’t create application infrastructure
• SRE makes sense at scale

Considerations when choosing team topologies

• Technical/Cultural Maturity
• Organization Size, Software Scale, Engineering Maturity
• Split responsibilities to break down silos
• Dependencies and wait times between teams

Chapter 5: The Four Fundamental Team Topologies

Stream-Aligned Teams

• continuous flow of work aligned to a business domain or organizational capability
• aligned to a single, valuable stream of work Could be:
  • product or service
  • single set of features
  • single user journey
  • single user persona Different Streams:
  • specific customer
  • business-area
  • geography
  • product
  • user-persona funded:
  • long-term
  • sustainable
• no handoffs to other teams

Capabilities

• App Security
• Commercial
• Design/Architecture
• Development/Coding
• Infra/Operation
• Metrics/Monitoring
• Product management and ownership
• Testing/quality
• UX

Stream-Aligned naming incorporates the sense of flow

Behaviors:

• produce steady flow of work
• course correct based on feedback
• learn and adapt
• zero handoffs
• evaluated on sustainable flow
• time and space for quality
• reaches out to fundamental-topologies teams
• Goal: autonomy, mastery, purpose

Enabling Teams

• Composed of specialists in a given technical domain and bridge capability gap

Behaviors:

• seek to understand need of stream-aligned teams
• stay’s on top of developments in their domain
• messenger of both good news and bad news
• proxy for services (internal/external)
• promotes learning

Complicated Subsystem Teams

• responsible for areas that require specialized knowledge and expertise

Behaviors

• mindful of current stage of development
• collaborates closely with stream-aligned teams
• higher delivery speed and quality

Platform Teams

• enable the stream-aligned teams to deliver work

Behaviors

• stream-aligned teams
• fast prototyping techniques and involves stream-aligned teams for feedback
• focus on usability and reliability
• leads by example
• understands adoption not immediate

Good platforms

• force multiplier
• standards
• templates
• APIs
• best practices

Thinnest Viable Platform (TVP)

• balance between keeping platform small and ensuring platform helps accelerate software delivery
• don’t let the platform dominate the discourse
• reduces complexity while exposing functionality

Platforms:

• built on a platform mindset (platforms all the way down)
• has users and defined active hours => reliability targets
• designed for long term user needs

Common Team Conversion

• Most teams should be long-lived, multidisciplined, stream-aligned teams
• Infra teams -> Platform Teams
• Components teams -> Platform or other teams
• Tooling Teams -> Enabling or Platform Teams
• Architecture team -> Enabling Team

Chapter 6: Choose Team First Boundaries

Overview:

• Monoliths can be hidden by team boundaries
• Consider software boundaries when necessary and suitable

Flow:

• difficult when teams interactions are complicated

Team First Approach to Software Responsibilities

software delivery problems

• unclear boundaries between teams & responsibilities
• tight coupling between teams

monolith -> loosely coupled services

• consider affect to teams

Hidden Monoliths

• Application Monoliths: single, large application w/ many dependencies
• Joined at the database Monoliths: multiple applications sharing a single database
• Monolithic Builds: one gigantic continious-integration to build a version
• Monolithic Release: smaller components bundled together into a release
• Monolithic Model: single domain langauage and representation
• Monolithic Thinking: one size fits all approach to team design
• Monolithic Workplace: single, open office layout for all teams

Software Boundaries or Fracture Planes

Fracture Plane: natural seam, allows system to be split into parts

• Business Domain Bounded Context: e.g. Domain Driven Design
• Regulatory Compliance: Split on sub-systems that are in different scopes of regulation
• Change Cadence: separate parts of the system that change at different rates
• Team Location: separate teams in different locations
• Risk: subsystems with different risk profiles
• Performance Isolation: subsystems with different performance requirements
• Technology: subsystems using different technologies
• User Personas: subsystems serving different user personas

Taxonomy

References:

GPT for better understanding my cognitive, emotional, and relational architecture

Overview

I recently came across a couple of posts on LinkedIn¹² that got me thinking about how I can use GPT to better understand my cognitive, emotional, and relational architecture. The post discusses how GPT can be used to create a mind map of your life, which can help you identify patterns and connections in your thoughts and feelings.

I decided to give it a try and see what I could come up with. The results were fascinating and provided me with a new perspective on myself.

The prompt I used was:

I want to better understand my cognitive, emotional, and relational architecture - how my mind works, how I process the world, and what makes me thrive. Based on our conversations, patterns in how I think, and the way I talk through problems, generate a structured table that maps out my internal wiring.

Categories should include:

Guiding Force
Core Personality
Superpowers
Growth Edges
Cognitive Architecture
Emotional Architecture
Relational Architecture

For each one, include a short description (how I experience it) and a ‘How It Shows Up’ column (real-life traits or behaviors that others might notice).

Make it sound like me. Use natural language, vivid but clear phrasing, and focus on insight, not fluff.

Results

Curious what you think! I found this exercise to be incredibly insightful and a great way to reflect on my personal development journey. If you’re interested in trying it out, I encourage you to give it a go and see what insights you can uncover about yourself.

References

Book Notes: Hands On Large Language Models: Language Understanding and Generation

Overview

This post contains my notes on the book *Hands-On Large Language Models: Language Understanding and Generation.

You can find the book on Amazon

I’ll be adding my notes to this post as I read through the book. The notes will be organized by chapter and will include key concepts, code examples, and any additional insights I find useful.

Chapter 1: An Introduction to Large Language Models

Chapter 1 introduces the reader to the recent history of Large Language Models (LLMs). The diagrams in this chapter are particularly useful for understanding the evolution of LLMs and how they relate to other AI technologies. It’s a great segway from the machine learning and neural networks covered in the previous book I’m reading in parallel.

Chapter 2: Tokens and Embeddings

Chapter 2 introduces the concept of tokens and embeddings.

Tokens are the basic units of text that LLMs use to process and generate language. The chapter covers a number of LLM Tokenizers including BERT (cased and uncased), GPD-2, FLAN-T5, StarCoder2, and a few others. It provides details on how each tokenizer works and how they differ from one another.

Token embeddings are numerical representations of tokens that capture their semantic meaning. Embeddings can be used to represent sentences, paragraphs, or even entire documents. Further, embeddings can be used in Recommendation Systems. The chapter covers a song recommendation system that uses embeddings to recommend songs based on a song input by the user.

Chapter 3: Looking Inside Large Language Models

Note: This chapter contains a number of useful diagrams that I’ve described in my own representation. However, the diagrams are not reproduced in their entirety. Please refer to the book for the complete diagrams and explanations.

Chapter 3 takes a deeper dive into the architecture of LLMs. We start out with a view into the Inputs and Outputs of Trained Transformer LLMs. This might be an overly simplified view, but it helps to understand the basic flow of data through an LLM.

The transformer generates a single output token at a time, using the previous tokens as context. This is known as an autoregressive model.

Diving a little deeper, we learn about the Transformer architecture. It’s composed of a Tokenizer, a stack of Transformer blocks, and an LM Head.

Going further, the tokenizer breaks down the input text into tokens and becomes a token vocabulary. The set of transformer blocks have token embeddings based on the token vocabulary. The LM head is a neural network layer that contains token probabilities for each token in the vocabulary.

Greedy decoding is when the model selects the token with the highest probability at each step.

It is possible to process in parallel multiple input tokens and the amount of tokens that can be processed at once is referred to as the context size. Keep in mind that embeddings are not the same as tokens, but rather a numerical representation of tokens that captures their semantic meaning.

Keep in mind that only the last token in the sequence is used to generate the next token. Even though this is the case, the processing stream results are parallelized and can be cashed to improve efficiency.

Digging even deeper, we learn about the Transformer blocks. Each block consists of a self-attention mechanism and a feed-forward neural network.

The feed-forward neural network is the source of learned information that enables the model to generate coherent text.

Attention is a key mechanism in LLMs that allows the model to focus on specific parts of the input sequence when generating text.

Taking a look into attention more closely, we see that we are getting to the core of how LLMs work.

Note: The book mentions projection matrices which are shown in the diagram below. However, it doesn’t explain them in detail. If you’re interested in understanding projection matrices, a good resource that I found helpful is this article. The content appears to be a summarization of the original paper on self-attention, Attention Is All You Need. Another good resource is The Illustrated Transformer

Using the queries and keys, the model calculates relevance scores for each token in the input sequence. The scores are then multiplied by the values to produce the output vectors.

Attention is a powerful mechanism that allows the model to weigh the importance of different tokens in the input sequence when generating text. Newer LLMs have available a more efficient attention mechanism called sparse attention, which can be strided or fixed. These attention mechanisms use fewer input tokens as context for self-attention.

Additionally, there are other attention mechanisms such as:

• Grouped Query Attention (GQA)
• Multi-Head Attention
• Flash Attention

Chapter 4: Text Classification

The goal of text classification is to assign a label to a piece of text based on its content. Classification can be used for a variety of tasks, such as:

• sentiment analysis
• topic classification
• spam detection
• intent detection
• detecting language

Techniques:

• Text Classification with Representation Models
• Text Classification with Generative Models

Text Classification with Representation Models

How it works:

1. Based models are fine-tuned for specific tasks, like classification or embeddings.

2. The models are fed inputs and outputs specific to the task are generated.

There are some suggestions for models that are good for text classification:

• BERT base model (uncased)
• RoBERTa base model
• DistilBERT base model (uncased)
• DeBERTa base model
• bert-tiny
• ALBERT base v2

When looking to generate embeddings the MTEB Leaderboard is a good resource to find models.

To evaluate the performance of classification models that have labeled data, we can use metrics such as accuracy, precision, recall, and F1 score.

Zero-shot classification is a technique that allows us to classify text without any labeled data.

Using the cosine similarity function, we can compare the embeddings of the input text to the embeddings of the labels.

Text Classification with Generative Models

Prompt engineering is the process of designing prompts that can effectively elicit the desired response from a generative model.

The T5 model is similar to the original Transformer architecture, using an encoder-decoder structure.

OpenAI’s GPT model training process is published here: https://openai.com/index/chatgpt/

Chapter 5: Text Clustering and Topic Modeling

Text clustering is the process of grouping similar pieces of text together based on their content, yeilding clusters of symantically similar text.

Text clustering can be used for topic modeling, which is the process of identifying the main topics in a collection of text.

The book example uses ArXiv papers as the text corpus.

Common Pipeline for Text Clustering:

1. convert input documents -> embeddings w/ embedding model
2. Reduce dimensionality w/ dimensionality reduction model
3. find groups of documents w/ cluster model

Dimensionality Reduction

There are well known method for dimensionality reduction including:

• Principal Component Analysis (PCA)
• Uniform Manifold Approximation and Projection (UMAP)

Clustering Algorithms

An example of clustering algorithms include:

• Hierarchical Density-Based Spatial Clustering of Applications with Noise (HDBSCAN)

Visualization of clusters can be done using tools like Matplotlib.

BERTopic: Modular Topic Modeling Framework

1. Follow the same procedure in text clustering to generate clusters
2. Model distribution over words, bag of words - use frequency of words in each cluster to identify topics
3. Use class-based term frequency inverse document frequency (c-TF-IDF) to identify words that are unique to each cluster

A full pipeline for topic modeling using BERTopic:

BERTopics can be used like Legos to build custom pipelines.

Chapter 6: Prompt Engineering

Basics of using text generation model

• Select a model considering:
  • opensource vs proprietary
  • output control

1. Choose opensource or proprietary model

suggestion: start with a small foundational model

2. Load the model

3. Control the output

• set do_Sample=True to use temperature and top_p

4. Tune Temperature and top_p for the use case

Intro to prompt engineering

Ingredients of a good prompt:

• When no instructions are given, the model will try to predict the next word based on the input text.
• Two components of basic instructions:
  1. Task description
  2. Input text (data)
• Extending the prompt with output indicator allows for specific output

Use cases for instruction based prompts:

• Supervised classification
• Search
• Summarization
• Code generation
• Named entity recognition

Techniques for improving prompts:

• Specificity
• Hallucination mitigation
• Order

Complex prompt components:

• Persona
• Instruction
• Context
• Format
• Audience
• Tone
• Data

In-context learning:

• Zero-shot learning
• One-shot learning
• Few-shot learning

Chain prompting:

• Break the task into smaller sub-tasks and use the output of one prompt as the input to the next prompt.
• Useful for:
  • Response validation
  • Parallel prompts
  • Writing stories

Reasoning with Generative Models

Chain of thought:

• Prompt the model to think step-by-step

Self-consistency:

• using the same prompt multiple times to generate multiple responses
• works best with temperature and top_p sampling

Tree of thought:

• useful when needing to explore multiple paths to a solution
• ask the model to mimic multiple agents working together to solve a problem
• question each other until they reach a consensus

Output Verification

• Useful for:
  • Structured output
  • Valid output
  • ethics
  • accuracy

Techniques:

• Provide examples of valid output

Grammar: constrained sampling

• use packages for:
  • Guidance
  • Guardrails
  • LMQL

Chapter 7: Advanced Text Generation Techniques and Tools

LangChain is a framework for using LLMs which has modular components. Phi-3 is an implementation of LangChain.

Quantization reduces the number of bits used to represent parameters of an LLM while retaining most of its information. It reduces precision of value without removing vital information.

A basic form of LongChain is a single chain.

Example: PromptTemplate -> LLM -> Output

More complex chains can have multiple inputs and outputs.

Example: Input -> PromptTemplate1 -> LLM1 -> Output1 -> PromptTemplate2 -> LLM2 -> Output2

The book uses a template chain to desmonstrate how to write a story using multiple prompts.

Example: Title -> Character -> Story

Limitation: LLMs don’t have memory of previous interactions.

To address this limitation, we can use a memory component to summarize previous interactions and then include the summary in the prompt for the next interaction.

Agents

Agents are systems that use an LLM to determine which actions to take based on the input and the current state of the system.

Agests use step by step reasoning:

• thought
• action
• observation

Chapter 8: Semantic Search and Retrieval-Augmented Generation

The book covers three broad categories for how langage models can be used for search:

• Dense Retrieval
• Reranking
• Retrieval-Augmented Generation (RAG)

Dense Retrieval

Using embeddings to represent documents and queries in a vector space.

Drawbacks

• Texts don’t have an answer => use threshild level for maximum distance to return an answer
• Exact match => use hybrid search (semantic + keyword)
• Corpus other than what the model was trained on => use domain-specific model
• limited context size => one vector per document or chunk documents

Options for chunking

• each sentence
• paragraph
• include title in chunk
• add text before and after chunk

Use vector database to store and search embeddings.

Reranking

Reranking can be used to improve the results of a search by using a more powerful model to rerank the results of a simpler model.

How Reranking works:

1. Query and Search results are passed to a reranking model
2. Reranking model scores each result based on relevance to the query
3. Results are sorted based on the scores

Effectively this is a classification problem where the model returns a value between 0 and 1 for each result.

Retrieval Evaluation Metrics

Evaluation metrics for retrieval models include:

• text archive
• set of queries
• relevance judgements

To compare two search systems, use the same query set and relevance judgements for both systems.

1. score only relevant documents

2. relevant docs (position 1) / (position 1)

3. continue for each relevant doc
4. average the scores for each query

Calculate Mean average precision (MAP) for each system and compare the results.

mean average precision (MAP) is the mean of the average precision scores for each query.

RAG: Retrieval-Augmented Generation

RAG systems reduce hallucinations and improve factuality.

Basic RAG Pipeline: Question -> Retriever -> Grounded Generation -> answer

Advanced RAG Techniques:

• Query Rewriting
• Multi-query RAG
• Multi-hop RAG
• Query routing
• Agentic RAG

RAG Evaluation Metrics:

• Fluency: how well the generated text flows and is easy to read
• Perceived Utility: how useful the generated text is to the user
• Citation recall: how well the generated text cites relevant sources
• Citation Precision: how accurately the generated text cites relevant sources
• Faithfulness: how accurate the generated text is
• Answer Relevance: how relevant the generated text is to the query

Chapter 9: Multimodal Large Language Models

A model that can process text and images is a multimodal model.

Vision Transformer (ViT) is a model that uses the Transformer architecture to process images.

ViT works by breaking an image into patches and then treating each patch as a token.

Multimodal embeddings can be used to represent both text and images in the same vector space.

CLIP is a model that can process both text and images.

How CLIP Training works:

1. Text and images are passed through separate encoders to generate embeddings
2. The similarity between sentence and image embeddings is calculated using cosine similarity
3. The text and image embeddings are trained to maximize the similarity between matching pairs and minimize the similarity between non-matching pairs

Creating a multimodal model requires a lot of power and data. One option is to use pre-trained models.

The Querying Transformer (Q-Transformer) is a model that can process both text and images.

Q-Transformer trains in two stages, one for each modality.

1. representation learning is applied for vision and language together
2. representations converted to soft prompts for a generative model

Then Q-Transformer is trained on:

• Image-text contrastive learning
• Image-text matching
• Image-grounded text generation

Use cases:

1. Image captioning
2. Multi-modal chat-based prompting

Chapter 10: Creating Text Embedding Models

Text Embedding models are the core of natural language processing applications.

Embedding Models

Unstructured text data => hard to process

convert text data => numerical vectors => easier to process, usable

semantic nature of text => similar meaning => similar vectors

What is contrastive learning?

Aims to teach models to differentiate between similar and dissimilar pairs of data points.

Ex. Why is this a cat and not a dog?

SBERT: Sentence-BERT is a modification of the BERT network that uses siamese and triplet network structures to derive semantically meaningful sentence embeddings.

Sentence transformers architecture uses a siamese network to generate sentence embeddings.

Creating an Embedding Model

1. Generate constrastive examples
2. Train the model
3. Evaluate the model

Generating Contrastive Examples

Start with a premise and generate positive and negative examples.

Use GLUE benchmark datasets to generate contrastive examples.

Training the Model

1. define an evaluator
2. use Semantic Textual Similarity Benchmark (STSB)
3. use the evaluator to get performance

For more in-depth testing, consider MTEB benchmark.

Use cosine similarity loss function to train the model.

Fine-tuning an embedding model

Supervised is the most straight forward way to fine-tune an embedding model.

Unsupervised approach: Transfomer-Based Sequential Denoising Auo-Encoder (TSDAE)

1. Add noise to the input text
2. Train the model to reconstruct the original text from the noisy input
3. Use the trained model to generate embeddings for new text data

Chapter 11: Fine-Tuning Representation Models for Classification

Supervised Classification

BERT & Feed-Forward Neural Network learn from one another during fine-tuning.

Few-Shot Classification

When labeled data is scarce, few-shot classification can be used to fine-tune a representation model for classification.

SetFit is a framework for few-shot classification that uses contrastive learning to fine-tune a representation model.

Continued Pre-training with Masked Language Modeling (MLM)

Continued pre-training with MLM can be used to adapt a representation model to a specific domain or task.

1. Pretraining from scratch
2. Continued pre-training from a pre-trained model
3. Fine-tuning for a specific task

Named Entity Recognition (NER)

Named Entity Recognition (NER) is the task of identifying and classifying named entities in text.

Chapter 12: Fine-Tuning Generation Models

Three LLM Training Steps: Pretraining, Supervised, Fine-tuning, and Preference Tuning

Steps to creating a high quality LLM:

1. Language Modeling Pre-train on one or more massive text datasets => produces a base model
2. Fine-Tuning 1 (Supervised Fine-Tuning or SFT) Aims to predict the next token based on an input that has additional labels.
3. Fine-Tuning 2 (Preference tuning) Final step to improve quality of model, align with AI Safety or human preferences.

Untrained -> Base -> Instruction Fine-Tuned -> Preference Tuned

Taxonomy

• Accuracy: A metric used to evaluate the performance of classification models, measuring the proportion of correct predictions.
• Attention: A mechanism that allows models to focus on specific parts of the input sequence, improving context understanding.
• Autoregressive Models: Models that generate text by predicting the next token in a sequence based on the previous tokens.
• Bag of Words (BoW): A simple representation of text that ignores grammar and word order but keeps track of word frequency.
• BERT (Bidirectional Encoder Representations from Transformers): A pre-trained model that uses a Transformer architecture to understand the context of words in a sentence.
• Byte Tokens: A tokenization scheme that represents text as a sequence of bytes, allowing for a more compact representation.
• Character Tokens: A tokenization scheme where each token represents a single character, useful for languages with complex morphology.
• Context Size: The number of tokens the model can consider at once when generating text, affecting its ability to maintain coherence.
• Embeddings: Numerical representations of words or tokens that capture their semantic meaning and relationships.
• F1 Score: A metric that combines precision and recall to evaluate the performance of classification models.
• Feed-Forward Neural Network: A type of neural network where connections between nodes do not form cycles, used in Transformer blocks.
• Flash Attention: An efficient attention mechanism that reduces memory usage and speeds up computation in LLMs.
• GPT (Generative Pre-trained Transformer): A type of LLM that is pre-trained on a large corpus of text and can generate coherent text based on a given prompt.
• Greedy Decoding: A text generation strategy where the model selects the token with the highest probability at each step.
• Grouped Query Attention (GQA): An attention mechanism that uses a single set of keys and values for multiple queries, improving efficiency.
• Inverse Document Frequency (IDF): is a measure of how important a word is to a document in a collection of documents.
• Large Language Models (LLMs): A type of AI model that is trained on large datasets to understand and generate human language.
• LM Head: The final layer of an LLM that generates the output tokens based on the processed input.
• Mean Average Precision (MAP): A metric used to evaluate the performance of information retrieval systems, measuring the average precision across multiple queries.
• Multi-Head Attention: An attention mechanism that allows the model to focus on different parts of the input sequence simultaneously.
• Output vectors: The numerical representations of the output tokens generated by the LLM.
• Parallel Processing: The ability to process multiple input tokens simultaneously, improving efficiency.
• Precision: The numerical accuracy of the computations performed by the model, affecting its performance and resource usage.
• Recall: A metric used to evaluate the performance of classification models, measuring the ability to identify all relevant instances.
• Self-Attention: A mechanism that allows the model to weigh the importance of different tokens in the input sequence when generating text.
• Sparse Attention: An efficient attention mechanism that uses fewer input tokens as context for self-attention, reducing computational complexity.
• Subword Tokens: A tokenization scheme where tokens can represent parts of words, allowing for better handling of rare or unknown words.
• T5 model: Text-To-Text Transfer Transformer, a model that converts all NLP tasks into a text-to-text format.
• Temperature: A parameter that controls the randomness of the model’s output, with higher values leading to more diverse text.
• Tokenization: The process of breaking down text into smaller units (tokens) for processing by LLMs.
• Token Embedding: The process of converting tokens into numerical vectors that capture their semantic meaning.
• Token Probabilities: The likelihood of each token in the vocabulary being the next token in a sequence, used for text generation.
• Top-p Sampling (Nucleus Sampling): A text generation strategy that selects tokens from the smallest set whose cumulative probability exceeds a threshold p, allowing for more diverse outputs.
• Transformer: A neural network architecture that uses self-attention mechanisms to process sequences of data, widely used in LLMs.
• Transformer Blocks: The building blocks of the Transformer architecture, consisting of layers of attention and feed-forward neural networks.
• Trained Transformer LLMs: LLMs that have been trained on large datasets using the Transformer architecture, enabling them to understand and generate human language effectively.
• Word Tokens: A tokenization scheme where each token represents a whole word.
• word2vec: A technique that uses neural networks to learn word embeddings, capturing semantic relationships between words.

References:

• Speech and Language Processing
• Attention Is All You Need
• Generating Long Sequences with Sparse Transformers
• Fast Transformer Decoding: One Write-Head is All You Need
• GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
• FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
• FlashAttention-2:Faster Attention with Better Parallelism and Work Partitioning
• The Illustrated Transformer
• On Layer Normalization in the Transformer Architecture
• Root Mean Square Layer Normalization
• GLU Variants Improve Transformer
• RoFormer: Enhanced Transformer with Rotary Position Embedding
• Efficient Sequence Packing without Cross-contamination: Accelerating Large Language Models without Impacting Performance
• Introducing packed bert for 2x faster training in natural language processing
• A Survey of Transformers
• Transformers in Vision: A Survey
• A Survey on Vision Transformer
• Open X-Embodiment: Robotic Learning Datasets and RT-X Models
• Transformers in Time Series: A Survey
• HuggingFace Datasets
• HuggingFace Cornell Rotten Tomatoes
• Scaling Instruction-Finetuned Language Models
• ArXiv
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
• Understanding searches better than ever before
• Bing delivers its largest improvement in search experience using Azure GPUs
• Introduction to Information Retrieval
• Evaluation in information retrieval
• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• UKPLab Sentence Transformers
• https://gluebenchmark.com/

Book Notes: In this Economy? How Money & Markets Really Work

Overview

This post contains my notes on the book In this Economy? How money & Markes Really Work.

You can find the book on Amazon

I’ll be adding my notes to this post as I read through the book. The notes will be organized by chapter and will include key concepts, code examples, and any additional insights I find useful.

Chapter 1: The Economic Kingdom

Chapter 1 introduces the concept of the “Economic Kingdom,” which is a framework for understanding how money and markets operate in our society.

The chapter discusses the following key concepts:

• Monetary Policy Castle: Manages the entire kingdom and is owned by the Federal Reserve. Generally in charge of inflation and the labor market.
• US Dollar Castle: Secret weapon of Monetary Policy Castle.
• The Commodity Castle: Basic goods used by everyone.

Contractionary monetary policy: things slow down, people don’t spend, people loose jobs Expansionary monetary policy: things speed up, people spend, people get jobs

Chapter 2: The Vibe Economy

Chapter 2 begins by explaining the relationship of households and businesses and the flow of resources.

Note: The book offers a diagram that illustrates the flow of money between households and businesses.

Households: Inputs: - Income - Buying Goods & Services Outputs: - Spending - Land, Labor, Capital

Businesses:
    Inputs:
        - Resources
        - Revenue
    Outputs:
        - Selling Goods & Services
        - Wages, Rent, Profit

Fuel Vibes:

Fuel: As gasoline prices rise, consumer sentiment tends to decrease.

Higher prices make us feel bad => if we feel bad, the economy feels bad

Oil Market influenced by: - how much oil is being produced

• how many people are consuming oil

Sentiment drives the economy:

• If people feel good, they spend more money
• If people feel bad, they spend less money

Sentiment is influenced by emotions. These emotions are summarized in varius theories:

• Prospect Theory
• Framing Effect
• Anchoring and Adjustment
• Endowment Effect
• Regret Theory
• Intertemporal Choice Theory
• Affective Forcasting
• Social Identity Theory
• Consumer Emotional Engagement

Reflexivity: The idea that the economy is influenced by people’s perceptions and beliefs about it, which can create a feedback loop.

1. Initial Perception/Believe
2. Market Action Based on Perception/Believe
3. Refliexive Feedback Loop
4. Bubble Bursts

Benefits of reflexivity to companies:

1. attracts investments based on expectations of future growth
2. gain critical edge in hiring top talent

What is sentiment?

Expectations Theory Reality

How we feel X How everyone else feels = Sentiment

Consumer Confidence Survey is an example of consumer sentiment measurement

Chapter 3: The Weird World of Money

What is Money?

• A store of value
• A unit of account
• A medium of exchange

Evolution of money:

• Growth of societies and contact with other civiliations led to the need for a common medium of exchange.
• Barter System: Direct exchange of goods and services without using money.
• Example: Is a bag of grain worth 15 cows or 10 cows?
• Coins solved the problem of barter by providing a standardized medium of exchange.

American Currency:

1700s: American colonies depend on European currencies (piece of eight) 1775: Continental Congress issues Continental Currency to fund the Revolutionary War 1785: Due to inflation, the Continental Currency is replaced by the US dollar

Hamilton’s Financial Plan:

• a federal bank that provides credit to the government and businesses
• issue a stable national currency
• safe place to store money

1791: First Bank of the United States is established 1811: First Bank of the United States charter expires, states issue their own currencies 1816: Second Bank of the United States is established Wildcat Banking Era: state-chartered banks issue their own currencies, leading to instability and bank failures 1863: National Banking Act creates a system of national banks and a national currency, only national banks can issue currency 1971: US dollar is no longer backed by gold, leading to fiat currency

The federal reserve enforces the promise and collective trus in the US dollar.

Chapter 4: The Mechanics of Modern Money

Banks:

• Gatekeepers of money
• Money business model

US Government:

• Creator of money
• Facilitator of what banks do

How is money created?

1. Issuing coins and notes
2. Through credit markets: issuance of government bonds

Banking Blueprint:

Built on:

• trust
• ability to borrow short (customer deposits)
• lend long (loans)

Fractional Reserve Banking: Banks are allowed to keep a fraction of deposits as reserves and lend out the rest.

• Banks use money, invest in securities, and make loans to earn interest.

Art of Lending: Banks hold a balance sheet of assets and liabilities.

• Assets: loans, securities, reserves
• Liabilities: customer deposits, borrowings
• Net Worth: assets - liabilities

Excess reserves: reserves held by banks above the required minimum.

Hedging: Banks use various strategies to manage risks, like interest rate fluctuations. Example:

• Interest rate swap: an agreement between two parties to exchange interest payments on a specified principal amount.

How banks fail:

insolvency: liabilities exceed assets illiquidity: unable to meet short-term obligations

Examples:

• Silicon Valley Bank (SVB): no hedges to protect against downside risk of rising interest rates
• 2008 Financial Crisis: use of Collateralized Debt Obligations (CDOs) and mortgage-backed securities (MBS) led to widespread defaults and bank failures.

Dollar’s Reign: Countries worldwide use the US dollar as a reserve currency.

• the dollar’s value goes up when there is uncertainty in the global economy as more people want to hold dollars.
• the dollar can be an inflation hedge as it tends to hold its value better than other currencies during inflationary periods.

Structure:

• least nasty alternative
• surplus and deficit national economies
• balance of payments

Chapter 5: Supply and Demand

low demand: no one wants it, price goes down high demand: everyone wants it, price goes up

Factors that influence demand:

• supply chain (issues)
  • example: semiconductor shortage
  • example: used car prices during the pandemic
  • example: egg shortage during the pandemic

presumed infinity of resources

Chapter 6: GDP and the Economy

Measuring GDP:

• GDP = C + I + G + (X - M)
  • C = Consumer Spending
  • I = Investment Spending
  • G = Government Spending
  • X = Exports
  • M = Imports

Consumption:

What people buy fueled by:

• income
• borrowing (credit cards)
• Savings

Investment: spending $$ for economic benefit over time Government Purchases: goods and services that government buys Net Exports: exports - imports

Nominal vs Real GDP: includes inflation vs adjusted for inflation

Increased borrowing can lead to increased nominal GDP.

Limitations of GDP:

• does not account for health and happiness
• does not account for environmental degradation

Real GDP per capita: total gdp / population

Productivity: ratio of output to input

Alternatives to GDP:

• degrowth
• ecological econmics
• postgrowth

Future of GDP:

• Big Growth Policies probably won’t work forever
• global challenges: climate change, income inequality, resource depletion rethinking of success; GDP may not be a holistic measure of well-being or social progress

Chapter 7: Commodities

What are commodities?

Commodities are raw materials used to create the building blocks of the economy.

Globalization + Free Trade & Comparative Advantage - Shipping issues & Floods & Fires

Oil: crude price is indicator of economy’s health Gas: direct impact to consumers as oil prices rise and fall

However: relationship between oil prices and gas prices is not symmetric

Gas Prices: Rocket and Feather Effect

• The market power of gas stations
• Fear of price variability

Metals: Steel, Aluminum, Copper Supply issues in one can affect the others

Renewables:

• We still need oil to build out EVs, solar panels, and nuclear reactors
• renewable isn’t just about tech
• problems to solve:
  • ease trade barriers
  • sharing tech in energy storage
  • commit to collective action

AI and Commodities:

• AI can help manage crops
• predict and locate resources

AI’s computational power is linked to raw material

Symbiotic Relationship

Chapter 8: Inflation

What is Inflation?

• Inflation is the decline in purchasing power of a currency over time.
• Inflation decline != price decline

Other types of “flation”:

• deflation: decline in prices
• hyperinflation: extremely high and typically accelerating inflation
• stagflation: high inflation combined with stagnant economic growth and high unemployment

Measuring Inflation:

• Consumer Price Index (CPI): measures the average change in prices paid by consumers for a basket of goods and services over time.
• Personal Consumption Expenditures (PCE): measures the average change in prices paid by consumers for goods and services, but includes a broader range of items than CPI.
• Whole Sale Price Index (WPI): measures the average change in prices received by producers for goods and services at the wholesale level.
• Producer Price Index (PPI): measures the average change in prices received by producers for goods and services at the producer level.
• GDP Deflator: measures the average change in prices of all goods and services included in GDP.

Causes of Inflation:

• Supply and Demand: when demand for goods and services exceeds supply, prices tend to rise.
• Broken Supply Chains: disruptions in supply chains can lead to shortages of goods and services, which can drive up prices.
• Monetary Policy: when central banks increase the money supply, it can lead to inflation

Stages of Inflation: Stability->Implulse->Propagation/Amplification->Conflict Wage Price Spiral: Higher Wage Demands->Firms increase Prices->Higher Inflation->Increase Inflation Expectations->Higher Wage Demands Inflation Narrative Path: Inflation -> Worried about Inflation -> Forecast Lower Sales -> Raise Prices -> Consumers feel worse

Deflation: Harmful as inflation, can cause economic slowdown. Hyperinflation: extremely high inflation, can lead to economic collapse and social unrest due to loss of confidence in currency.

Taxonomy

• Fiat Money: currency that is not backed by a physical commodity, such as gold or silver, but rather by the government that issues it.
• Inflation: rise in prices that create a decrease in purchasing power.
• OPEC: Organization of the Petroleum Exporting Countries, a group of oil-producing countries that coordinate their oil production and pricing.
• Sentiment: the overall attitude or feeling of consumers and businesses towards the economy, which can influence spending and investment decisions.
• Prospect Theory: a behavioral economic theory that describes how people make decisions based on perceived gains and losses rather than absolute outcomes.
• Framing Effect: a cognitive bias where people react differently to the same information depending on how it is presented or framed.
• Anchoring and Adjustment: a cognitive bias where people rely too heavily on the first piece of information they receive (the “anchor”) when making decisions, and then make adjustments based on that anchor.
• Endowment Effect: a cognitive bias where people value something more highly simply because they own it, leading to a reluctance to sell or trade it.
• Regret Theory: a behavioral economic theory that suggests people anticipate regret when making decisions and may avoid choices that could lead to regret, even if those choices are rational.
• Intertemporal Choice Theory: a behavioral economic theory that examines how people make decisions about trade-offs between immediate and delayed rewards, often leading to preferences for immediate gratification over long-term benefits.
• Affective Forcasting: a cognitive bias where people predict their future emotional states based on current feelings, which can lead to inaccurate predictions about how they will feel in the future.
• Social Identity Theory: a psychological theory that explains how individuals derive their self-concept and identity from their group memberships, influencing their behavior and decision-making.
• Consumer Emotional Engagement: the emotional connection and involvement that consumers have with a brand or product, which can influence their purchasing decisions and loyalty.
• Comparative Advantage: the ability of a country to produce a good or service at a lower opportunity cost than another country.

References:

• Consumer Confidence Board

Book Notes: AI and Machine Learning for Coders - A Programmer's Guide to Artificial Intelligence

Overview

This post contains my notes on the book AI and Machine Learning for Coders - A Programmer’s Guide to Artificial Intelligence by Laurence Moroney[^1]. The book is a practical guide to building AI and machine learning applications using TensorFlow and Keras. It covers the basics of machine learning, deep learning, and neural networks, and provides hands-on examples of how to build and deploy AI applications.

You can find the book on Amazon

I’ll be adding my notes to this post as I read through the book. The notes will be organized by chapter and will include key concepts, code examples, and any additional insights I find useful.

Chapter 1: Introduction to tensorflow

In chapter 1 you’ll learn about the limitations of traditional programming and get some initial insight into Machine Learning. You’ll learn how to install tensorflow, in my case on a M2 Mac.

I searched around a little to see what common issues occurred for mac installations and came across a handy blog that covered different tensorflow installation options and included a handy script to verify tensorflow was indeed using my gpu. After a brief hiccup I added the following packages to my installation, re-ran the script and was on my way.

pip install tensorflow
pip install tensorflow-macos tensorflow-metal

At the end of this chapter you’ll build and train your first model, a simple linear regression model that predicts the output of a linear equation.

Note: Using the coding samples located in GitHub will make following along really easy. https://github.com/lmoroney/tfbook

Chapter 2: Introduction to Computer Vision

Using the Fashion MNIST dataset, chapter 2 introduces the reader to Neural Network design. Using a real, but simiple dataset, you’ll learn how to build a neural network that can classify images of clothing.

Chapter 3: Convolutional Neural Networks

In chapter three, you’ll explore Convolutional Neural Networks using images of humans and horses. You’ll use training and validation data to build up a model as well as learn about image augmentation to broaden the data and reduce overspecialization. Additional concepts introduced include Transferred Learning, Multiclass Classification, and Dropout Regularization.

Chapter 4: Using Public Datasets with TensorFlow Datasets

Using the TensorFlow Datasets library, chapter 4 introduces the reader to ETL which is a core pattern for training data. The chapter covers a practical example as well as how to use parallelization ETL to speed up the process.

Chapter 5: Natural Language Processing

Chapter 5 introduces the reader to tokenization, taking text and breaking it down into smaller units (tokens) for processing. It covers basics like Turning sentences into tokens, padding sequences, as well as more advanced techniques like removing stop words, and text cleaning.

The examples in this chapter use the IMDB, emotional sentiment, and scarcasim classification datasets as examples for building datasets from html like data, csv files, and json.

Chapter 6: Making Sentiment Programmable Using Embeddings

Chapter 6 uses a Sarcasm dataset to introduce the reader to word embeddings. Words are given a numerical representation of positive numbers that represent Sarcasm and negative numbers that represent realistic statements. Sentences could then be represented as a series of these numbers and evaluated for a Sarcasm score.

The example in this chapter begins to analyze accuracy and loss against the training and validation datasets. This can help identify overfitting, which is when the model becomes overspecialized to the training data.

The example goes on to cover various techniques to improve the model. These include:

• Adjusting the learning rate
• Adjusting the vocabulary size
• Adjusting the embedding dimension

When these techniques are applied, finer tuning of the model can be achieved by:

• Using dropout
• Using regularization

Chapter 7: Recurrent Neural Networks for Natural Language Processing

Chapter 7 introduces Recurrent Neural Networks (RNNs) and how they can be used for Natural Language Processing tasks.

They provide a diagram of a recurrent neuron which shows how a recurrent neuron is architected.

Long Short-Term Memory (LSTM) is a type of RNN that is capable of learning long-term dependencies.

Bidirectional LSTMs are a type of LSTM that can process data in both directions, which can be useful for tasks like sentiment analysis where context from both the past and future can be important.

The exercise in this chapter reuses the exercise from chapter 6 and introduces stacked LSTMs.

Overtraining occurs in the example, so optimization techniques are applied to improve the model. These include:

• Adjust the learning rate
• Using dropout in the LSTM layers

A second example in this chapter uses pretrained embeddings, the GloVe set. In this second example, after the model is downloaded, an exercise to determine how many of the words in the corpus are in the GloVe vocabulary is performed.

Chapter 8: Using TensorFlow to Create Text

This chapter starts out with an example of tokenizing text and creating a word index. Then a model is built so that it can be trained.

With a trained model it can be used to predict the next word in a seqence. We can then use seed text to generate a token and test the model. We can then repeat this process to generate mode text with alternate seed text.

Using the same process we can use a differet, larger dataset to generate more complex text. The example adjusts the model slightly by adding an additional LSTM layer and increasing the number of epochs.

Chapter 9: Understanding Sequence and Time Series Data

Common attributes of time series data include:

• Trend
• Seasonality
• Autocorrelation
• Noise

Techniques for predicting time series data include:

• Naive Prediction to create a baseline

Measuring Prediction accuracy:

• Mean Absolute Error (MAE)
• Mean Squared Error (MSE)

Less Naive: Using Moving Averages

Chapter 10: Creating ML Models to Predict Seqeuences

Windowed datasets can be used to emulate a time series dataset. The example in this chapter uses a window size of 5 to predict the next value in the sequence.

It goes through the process of creating a windowed version of time series data, then building and training a model.

Tuning the model can be done through the use of Keras Tuner.

Chapter 11: Using Convolutional and Recurrent Methods for Sequence Models

This chapter continues the work from chapter 10 and introduces the use of Convolutional Neural Networks (CNNs) for sequence modeling.

It goes over the parameters of the Conv1D layer and how they can be used in the model.

It then goes on to use NASA weather data and using RNNs for seqence modeling and some tuning approaches.

This includes:

• Using Other recurrent methods, like gated recurrent units (GRU) and long short-term memory layers (LSTM)
• Using Dropout
• Using bidirectional RNNs

Chapter 12: An introduction to tensorflow lite

TensorFlow lite is a mobile version of TensorFlow.

The book walks through the process of converting a model to TensorFlow lite and then using it.

Chapter 13: Using TensorFlow Lite in Android Apps

This chapter walks through the process of using TensorFlow lite in an Android app.

I didn’t spend much time on this chapter as I don’t do Android development.

Chapter 14: Using TensorFlow Lite in iOS Apps

This chapter walks through the process of using TensorFlow lite in an iOS app.

I didn’t spend much time on this chapter as I don’t do iOS development.

Chapter 15: An Introduction to TensorFlow.js

This chapter walks through the process of using TensorFlow.js and how you can train models in the browser.

I didn’t spend much time on this chapter.

Chapter 16: Coding Techniques for Computer vision in TensorFlow.js

This chapter was another chapter that spend time on brower-based image classification.

I didn’t spend much time on this chapter.

Chapter 17: Reusing and Converting Python Models to JavaScript

This chapter walks through the process of converting python-created models and converting them to TensorFlow.js.

I didn’t spend much time on this chapter.

Chapter 18: Tranfer Learning in JavaScript

This chapter walks through the process of using transfer learning in TensorFlow.js.

I didn’t spend much time on this chapter.

Chapter 19: Deployment with TensorFlow Serving

This chapter walks through the process of using TensorFlow Serving to deploy models.

I didn’t spend much time on this chapter.

Chapter 20: AI Ethics, Fairness, and Privacy

This chapter covers some of the ethical considerations when using AI and machine learning.

• Fairness
• Facets
• Federated Learning

Taxonomy

• Autocorrelation: The correlation of a signal with a delayed copy of itself as a function of delay.
• Convolution: Mathematical filter that works on the pixels of an image.
• Dropout: A regularization technique that randomly sets a fraction of input units to 0 at each update during training time, which helps prevent overfitting.
• Embedding Dimension: The size of the vector representation of each word in the vocabulary.
• Learning Rate: A hyperparameter that controls how much to change the model in response to the estimated error each time the model weights are updated.
• Long Short-Term Memory (LSTM): A type of recurrent neural network (RNN) architecture that is capable of learning long-term dependencies.
• Natural Language Processing (NLP): A field of AI that focuses on the interaction between computers and humans through natural language.
• Noise: Random variation in data that does not contain useful information.
• Out of Vocabulary (OOV): Words that are not present in the training vocabulary.
• Overfitting: When the model becomes overspecialized to the training data.
• Padding: Adding zeros to the beginning or end of a sequence to make it a fixed length.
• Regularization: Techniques used to prevent overfitting by adding a penalty to the loss function based on the complexity of the model.
• Seasonality: A pattern that repeats at regular intervals in the data.
• Stop Words: Common words that are often removed from text data to reduce noise and improve model performance.
• Tokenization: The process of breaking down text into smaller units (tokens) for processing.
• Transfer Learning: Taking layers from another architecture.
• Trend: A long-term increase or decrease in the data.
• Vocabulary Size: The number of unique words in the training dataset.

References:

• Understanding LSTM Networks
• GloVe: Global Vectors for Word Representation

Pragmatism for decision-making in Software Development

Overview

This post discusses pragmatism as a tool in Software Development. I consider myself to be pragmatic in my approach to software engineering, and wanted to explore the concept a little more.

What is pragmatism?

Pragmatism¹ is a philosophical tradition that began in the United States around 1870. It is a way of thinking that focuses on the practical consequences of actions rather than on abstract principles. Pragmatists believe that the truth of an idea is determined by its usefulness in solving real-world problems. They are concerned with what works, rather than with what is theoretically correct.

Pragmatism in software development

Pragmatism can be a powerful tool in software development. Software projects can be complex, with lots of moving parts, and there’s lots of opportunity for things to go wrong, extending timelines and risking value delivery. Pragmatism can help you make decisions that are practical and effective, rather than getting bogged down in theoretical debates.

Some goals for pragmatism in software development can be distilled into the following:

• deliver value to the customer (users and/or organization)
• maximize stakeholder satisfaction
• minimize stakeholder dissatisfaction

Looking around for inspiration

I looked around to see if there’s any existing work on pragmatism in software development. I found a few interesting papers, articles, and books that I wanted to share.

Optimization in Software Engineering - A Pragmatic Approach

Guenther Ruhe² published a paper on Optimization in Software Engineering - A Pragmatic Approach³. The paper takes a process-based approach and includes a checklist for performing the optimization process. This process includes:

• Scoping and Problem analysis
• Modeling and Problem formulation
• Solution Design
• Data collection
• Optimization
• Validation
• Implementation
• Evaluation

The following describes each step in the process:

Scoping and Problem analysis

The first and most obvious step in the process is to ask if the problem can be solved easily. Easily solved problems help sidestep the need for additional time investment. When looking for an easy solution, consider alternatives as well.

Understand the stakeholders and decision makers around the problem. How important is this to them? How much time and effort can be invested in solving the problem? What’s the budget associated to solving the problem, considering both time and money?

Which approach best aligns with the business objectives and how would optimization benefit the problem context?

Modeling and Problem formulation

Depending on the complexity of the problem, work to break it down into smaller, more manageable parts. Identify key variables, constraints, and any dependencies.

Model project resourcing, budget and time for each phase of the project.

Identify technological constraints and dependencies.

Solution Design

Is there a solution already available that can be used as a baseline? If so, how does the proposed solution compare to the baseline?

What are the perceived expectations for the optimized approach?

Data collection

What data is available and what data is needed to solve the problem?

How reliable is the data? Is there a need for data cleaning?

Optimization

What settings are made and why? How do they vary?

Validation

What are the criteria for validation? How are they measured?

Do the stakeholders agree with the proposed solution?

Implementation

Is there anything that needs to be adjusted?

Evaluation

How much does the implemented solution solve the original problem and is acceptable by the stakeholders?

How much above the baseline does the implementation improve?

The Pragmatic Programmer

The Pragmatic Programmer is a book by Andrew Hunt and David Thomas. The book is a guide to software development that focuses on practical advice and best practices. The authors emphasize the importance of writing clean, maintainable code, and of being pragmatic in your approach to software development.

There are so many good nuggets in this book. If this isn’t already on your bookshelf, I highly recommend it.

Some of my favorites nuggets include:

• Don’t live with broken windows
• Good-Enough Software
• Remember the bigger picture
• Prototype to learn

Further reading on Pragmatism

• https://blog.liblab.com/pragmatic-engineers-philosophy-first-part/
• https://arxiv.org/pdf/1912.02071
• https://pkritiotis.io/technical-decision-making-fast-paced-environments/

Wrapping it up

I’m still exploring the concept of pragmatism in software development. Going beyond just pragmatism, I’m also interested in how to be a better product engineer⁴ while practicing pragmatism. I’m looking forward to sharing more on this topic in the future.

Introducing RFCs to Share Ideas

Overview

There are a lot of positive benefits of being on a remote team. Finding ways to connect with your team and build relationships is important. One way to do this is to share your ideas and have discussions about various design topics. This is a great way to learn from your peers and to share your knowledge with them. It’s also a great way to build trust and a sense of community through the activity of writing and healthy discussion with your peers.

In addition to connecting with your team and building relationships, sharing your ideas is a great way towards building a writing culture. Writing is a great way to document your thought process and to share it with others. For me, writing is a way to become more concise with my thoughts and to be more clear in my communication.

One approach to sharing ideas is to write an RFC (Request for Comments). This is a document that outlines a problem and a proposed solution. It’s a great way to get feedback on your ideas and to build consensus around them.

What is an RFC?

Per Wikipedia¹:

In 1969, Steve Croker invented the RFC system. He did this to create a way to record unofficial notes on the development of ARPANET. RFCs have since become official documents of internet specifications, communications protocols, procedures, and events.

There are so many great resources on how to write an RFC. This post from the Pragmatic Engineer is a great place to start and lists a lot of great resources on the topic.

I’ve come to know RFCs as a templated format for sharing ideas and seeking consensus. They range from very formal to informal. They can be used for a variety of things, such as proposing a new feature, discussing a problem, or documenting a decision.

How to Write an RFC

There are plenty of resources on how to write an RFC as they’ve been around for a while. Here are a few different formats I’ve come across and am interested in learning more about and trying:

• https://www.rfc-editor.org/rfc/rfc7322
• https://www.industrialempathy.com/posts/design-docs-at-google/
• Google Docs RFC Template

Keep Track of Your RFCs

Keeping track of the status for each RFC is important. This could be as simple as a spreadsheet, a more formal system like GitHub issues. The idea is to have a way to track the status of each RFC and to make sure that they’re being reviewed and acted upon. Keep your RFCs organized and easy to find. This could be as simple as a folder in Google Drive or in a GitHub repository.

• Draft
• Review
• Approved
• Discarded
• Deprecated

Sample RFC

Title: Using RFCs to Share Ideas
Authors:
John Costa

1 Executive Summary
The primary problem this RFC proposal is solving is how to arrive at a consensus. Documenting architecture decisions
would be done elseware.

2 Motivation
Often times there's a lot of great ideas that come up in discussions.  Unfortunately, these ideas never get documented
and are lost. Without a semi-formal process, it's easy for ideas to get lost. This is a great way to document your
thought process and to share it with others.

3 Proposed Implementation
The following proposal is a simplified version of a Request for Comment process based on the following published
resource. Much inspiration for this proposal and in some cases whole segments have been drawn from these resources:

* https://cwiki.apache.org/confluence/display/GEODE/Lightweight+RFC+Process
* https://philcalcado.com/2018/11/19/a_structured_rfc_process.html

Collaboration
Comments and feedback should be made in the RFC document itself. This way, all feedback is in one place and can be
easily referenced or referred to.

Authors must address all comments written by the deadline. This doesn't mean every comment and suggestion must be
accepted and incorporated, but they must be carefully read and responded to. Comments written after the deadline may be
addressed by the author, but they should be considered as a lower priority.

Every RFC has a lifecycle. The life cycle has the following phases:

* Draft: This is the initial state of the RFC, before the author(s) have started the discussion and are still working on the proposal.
* Review: This is the state where the RFC is being discussed and reviewed by the team.
* Approved: This is the state where the RFC has been approved and is ready to be implemented. It does not mean that the RFC is perfect, but that the team has reached a consensus that it is good enough to be implemented.
* Discarded: This is the state where the RFC has been discarded. This can happen for various reasons, such as the proposal being outdated, the team not reaching a consensus, or the proposal being too risky.
* Deprecated: This is the state where the RFC has been deprecated. This can happen when the proposal has been implemented and is no longer relevant, or when the proposal has been replaced by a better one.

Approval
The proposal should be posted with a date by which the author would like to see the approval decision to be made. How
much time is given to comment depends on the size and complexity of the proposed changes. Driving the actual decisions
should follow the lazy majority approach.

Blocking
If there are any blocking issues, the author should be able to escalate the issue to the team lead or the team. A block
should have a reason and, within a reasonable time frame, a solution should be proposed.

When to write an RFC?
Writing an RFC should be entirely voluntary. There is always the option of going straight to a pull request. However,
for larger changes, it might be wise to de-risk the risk of rejection of the pull request by first gathering input from
the team.

Immutability
Once approved the existing body of the RFC should remain immutable.

4 Metrics & Dashboards
There are no explicit metrics or dashboards for this proposal. The RFC process is a lightweight process that is meant to
be flexible and adaptable to the needs of the team.

5 Drawbacks
- Slow: The RFC process can take time
- Unpredictable: The rate of new RFCs is not controlled
- No backpressure: There is no mechanism to control the implementation of RFCs
- No explicit prioritization: RFCs are implicitly prioritized by teams, but this is not visible
- May clash with other processes: RFCs may not be needed for smaller things
- In corporate settings, the RFC process should have a decision-making process that is clear and transparent

6 Alternatives
- ADRs (Architecture Decision Records)
- Design Docs
- Hierarchical, democratic, or consensus-driven decision-making

7 Potential Impact and Dependencies
The desired impact of this proposal is to have a more structured way to share ideas and to build consensus around them.

8 Unresolved questions
- None

9 Conclusion
This RFC is a proposal for a lightweight RFC process and can be used for remote teams looking to build consensus around
ideas.

References

The following are some references that I’ve found useful in my research:

• https://www.rfc-editor.org/rfc/rfc7322
• https://newsletter.pragmaticengineer.com/p/software-engineering-rfc-and-design
• https://leaddev.com/technical-decision-making/thorough-team-guide-rfcs
• https://stackoverflow.com/questions/2302290/how-to-write-and-propose-an-rfc
• https://blog.pragmaticengineer.com/scaling-engineering-teams-via-writing-things-down-rfcs/
• https://cwiki.apache.org/confluence/display/GEODE/Lightweight+RFC+Process
• https://candost.blog/how-and-why-rfcs-fail/

Reviewing Code

Overview

Code reviews are a critical part of the software development process. They help to ensure that the code is of high quality, that it’s maintainable, and that it’s secure. They also help to ensure that the code is in line with the company’s goals and values. Code reviews are also a great way to learn from your peers and to share your knowledge with them.

Knowledge Sharing

The code review should be a learning opportunity for everyone involved, this could mean as part of the review or historically when looking back at motivations and decisions.

Higher Quality

The code review should ensure that the code is of high quality. This means that it should be free of errors and warnings, that it should run properly, and that it should accomplish the feature(s) it was designed to accomplish.

Better Maintainability

The code review should ensure that the code is maintainable. This means that it should be easy to read and understand, that it should be well-documented, and that it should follow coding and technical standards.

Increased Security

The code review should ensure that the code is secure. This means that it should be free of security vulnerabilities, that it should not introduce any new security vulnerabilities, and that it should follow security best practices.

Optimization Opportunities

The code review should consider if the code is efficient, not wasting resources, and is scaleable.

Assumptions

After the first pass through this blog post, I realized while writing this, there’s a few assumptions about the environment that I’m making.

One is that a version control system is being used and that the code is being reviewed in a pull request. This assumes healthy use of a version control system.

Another is that the code is being reviewed by teammates who you work closely with and that you trust to give and receive feedback and with positive intent.

Priorities

To follow the principles above, I try to review code with the following priorities in mind:

1. Is the code functional?

The first thing I try to do is understand if it accomplishes the feature(s) it was designed to accomplish. As a reviewer, this could mean reading a README and running the code. When running the code, I try to capture not only the happy path but also the edge cases and error handling. As a submitter, this could mean providing these tools for the reviewer, ideally as unit tests and README documentation.

2. Is the code clean and maintainable?

Secondly, I try to look at the code from cleanliness and maintainability perspective. To avoid as much subjectivity as possible, automated linters and static analysis tools should be used. In addition to these tools, the code should be well-documented, considering CSI (Comment Showing Intent)¹ standards. The CSI Standard should exist alongside Self-Commenting² Code practices, not instead of. The code should also have binaries and unnecessary cruft removed.

3. Is the code secure?

Thirdly, I try to look at the code from a security perspective. Admittedly, this is an area I’m learning more about. With that said, I delegate much of this to automated tools which cover things like OWASP® Top 10 and CWE/SANS Top 25.

4. Can the code be optimized?

Lastly, I try to look at the code from an optimization perspective. This means that the code should be efficient and not waste resources. It should also be scalable.

Design and architecture

Something I’ve been trying to do more of is using an RFCs (Request for Comments) ahead of writing code for larger changes. I think about the design and architecture of the code. This is a great way to get feedback on the design and approach well before the code is written. This is also a great way to get buy-in from the team on the approach.

Additional Considerations

Google’s Standard of Code Review mentions that the primary goal of the code review is to ensure that “the overall code health of Google’s codebase is improving over time”. This might be good for a big company like Google, but I feel that if you prioritize people over code, the code will naturally improve over time. This is why I like the idea of using code reviews as a learning and knowledge sharing opportunity.

Additionally, something that resonated with me from How to Do Code Reviews Like a Human (Part One), is that code reviews should be about the code, not the person. To help avoid some pitfalls use these techniques mentioned in the post:

1. never say “you”
2. Frame feedback as requests
3. Tie notes to principles, not opinions.

Checklist

The following is a checklist that’s hopefully useful for pull requests. The idea is to use these to be consistent in process and should be applicable for both the openers and reviewers.

Checklist:

• How

  • Does the code comply with the RFC (Request for Comments), if one exists?
  • Does the code accomplish the feature(s) it was designed to accomplish?
  • Is there documentation? README, CSI, Self-Commenting Code?

• What

  • Are there tests? Do they cover the happy path, edge cases, and error handling?
  • Are linting and static analysis tools being used? If so, are they passing?
  • Are there any security vulnerabilities? Is the project up to date?
  • Are there any optimization opportunities?
    • Are there opportunities to reduce redundant code (DRY?
    • Does it follow SOLID principles?

• Follow-Up/TODOs

  • Are there any follow-up items that could be addressed?

• Feedback

  • Is the feedback framed as a request?
  • Is the feedback tied to principles, not opinions?
  • Does the feedback avoid using “you”?

References

5x15 Reports to Advocate for the Work of Yourself, Project, or Team

Overview

I’ve found this less in smaller companies, but sometimes in larger companies, colleagues will take credit for the work of others. This is a toxic behavior that can lead to a lack of trust and a lack of collaboration. It’s important to recognize the work of others and to give credit where credit is due.

While this wouldn’t be the only reason for doing so, one of the solutions I’ve found to help erode the toxic behavior is to use 5x15 reports to advodacte for the work of one’s self, project, or team.

5x15

The 5x15 report is a weekly report that is sent to your manager. Yvon Chouinard, founder and CEO of outdoor equipment company Patagonia®, devised the 5-15 Report in the 1980s.¹ As the name implies, it should take no longer than 5 minutes to read and no more than 15 minutes to write.

If you get a chance to read about Yvon Chouinard, you’ll find that he’s a very interesting person. He’s a rock climber, environmentalist, and a billionaire. He’s also the founder of Patagonia, a company that is known for its environmental advocacy.²

How to Write a 5x15 Report

The 5x15 report is a simple report that is sent to your manager. As mentioned, it should be no longer than 5 minutes to read and no more than 15 minutes to write. The report should include the following:

1. Accomplishments: What you’ve accomplished in the past week.
2. Priorities: What you plan to accomplish in the next week.
3. Challenges: Any challenges you’re facing.
4. Stats: Your personal stats for the week.

Accomplishments

The accomplishments section is where you can advocate for the work of yourself, your project, or your team. This is one of the most critical sections of the 5x15 report. It’s important to recognize the work of others and to give credit where credit is due, which includes crediting yourself, your project, or your team.

Depending on the size of your accomplishments, try to size them in terms of the impact they’ve had on the company. For example, if you’ve saved the company $100,000, then you should mention that in your report. If you don’t know the impact of your accomplishments, then you should try to find out. Perhaps put this in the challenges section of your report.

In addition to charting the impact of your accomplishments, you could also frame them in terms of the company’s goals and values. For example, if your company values Execution, then you could frame your accomplishments in terms of how they’ve helped the company execute on its goals. As an engineer, I sometimes forget about the soft skills that are required to be successful in the workplace. The 5x15 is a great way to highlight where you’ve used these soft skills to be successful.

Priorities

This should be a list of your priorities for the next week. This is a great way to set expectations with your manager and provide an opportunity to change the priorities if they’re not aligned with the company’s goals and values at that time.

In general, priorities shouldn’t change too much from week to week. If they do, then you should try to find out why.

Challenges

This isn’t a section to complain about your job. This is a section to highlight the challenges you’re facing and to provide an opportunity for your manager to help you overcome them. If you’re not facing any challenges, perhaps you’re not pushing yourself, project, or team. Try to provide potential solutions to the challenges you’re facing along with the challenges themselves. This will show that you’re proactive and that you’re thinking about how to overcome the challenges you’re facing without needing to be told what to do.

Stats

This is a section to provide your personal stats for the week. These stats should be meaningful to you and your manager and probably something like Energy, Credibility, Quality of Life, Skills, and Social Capital as resources. This should be your dashboard if you were to have one.³

Something new I’m thinking about trying is to include more insight into how I’m pacing myself. Stretching, Executing, Coasting could be three states of flow for the given week.⁴ Personally, I would find this useful to know if my reports where Stretching or Coasting as this would be a queue on whether they have additional capacity to take on more.

Follow-up

Once you’ve established the process of sending 5x15 reports, you should check in with your manager to see if they’re finding them useful. If they’re not, then you should try to find out why. If they are, then you should try to find out how you can make them more useful. Depending on the feedback, you might need to adjust the format or the content of the report.

Summary

The 5x15 report is a great way to communicate with your manager and can be used to set the agenda for your 1:1s. If it’s not already part of your company’s culture, then I would recommend trying to introduce it. It’s a great way to advocate for the work of yourself, your project, or your team.