On improving coding and theming analysis

Collated and edited by Ross Woods, 2026

This is one of the weakest areas of qualitative research methodology. Compared with quantitative methods, coding and theme development are often under-proceduralised. Many textbooks explain what coding is, but provide relatively little guidance on how to do it systematically enough that another competent researcher could reasonably reproduce the process. This creates substantial scope for methodological innovation.

The biggest weakness is the "black box" problem. Currently, many qualitative papers effectively say: “I read the transcripts several times...” “I developed codes...” “These became themes...” That may represent weeks of work condensed into three sentences, and much of the reasoning remains invisible. One could argue that thematic analysis still contains a large "black box" between raw transcripts and published themes. Opening that black box is probably the greatest opportunity for methodological improvement.

1. More explicit coding procedures

Many guides leave dozens of important decisions unspecified; they simply say: “Read the transcript. “ “Generate initial codes.” “Group them into themes.”

A more procedural approach would make qualitative analysis far more transparent by specifying:

• how many transcripts are coded before revising the codebook
• when new codes are permitted
• when existing codes should be merged
• when codes should be split
• criteria for deleting codes
• maximum desirable code overlap
• decision rules for ambiguous passages
• stopping rules

2. Better rules for theme development

Perhaps the greatest weakness is that theme generation is often presented almost as intuition. Researchers are told to "Look for patterns." But what constitutes a pattern?

Themes could be evaluated against explicit quality criteria rather than researcher judgement alone. Possible innovations include explicit criteria such as:

• answer part of the research question
• appear across multiple participants
• possess conceptual coherence
• be distinguishable from other themes
• explain meaningful variation
• be supported by sufficient evidence
• have clear boundaries

3. Decision logs

Coding decisions often disappear. Researchers might instead create an auditable analytical trail by maintaining:

• why a code was created
• why it was changed
• why two codes were merged
• why one code was deleted
• why a quotation was assigned to multiple codes

4. More hierarchical coding systems

Many studies produce flat lists of codes. Instead, coding could better reflect the development of theory by routinely distinguishing between:

Level 1: Descriptive observations
↓
Level 2: Interpretive concepts
↓
Level 3: Broader theoretical categories
↓
Level 4: Explanatory propositions

5. Explicit code quality measures

Showing how analysis evolved would greatly strengthen transparency. Few researchers evaluate whether a code is actually good. Each code could receive a quality score. Possible criteria include:

• clarity
• uniqueness
• conceptual precision
• usefulness
• explanatory power
• frequency
• richness
• relationship with research questions

6. Tracking analytical development

Most theses present only the final themes. They rarely show:

Version 1
↓
Version 2
↓
Version 3
↓
Final framework

7. Better handling of contradictory data

Researchers often force contradictory quotations into existing themes. Alternative procedures would produce much more balanced analysis by requiring every theme to document supporting evidence, contradictory evidence, exceptions, and boundary conditions.

8. Computer-assisted coding beyond storage

Current software mainly helps organise codes, although the researcher would remain responsible for final decisions. Future systems could actively assist by:

• suggesting possible merges
• identifying duplicate codes
• flagging inconsistent coding
• identifying uncoded material
• detecting code drift
• showing evolving code networks

To implement such software for academic purposes, it needs to meet the standards for any other software used in academia, such as publishing its methods and limitations, and a reliable system for validating its results.

9. Quantifying qualitative analysis (carefully)

Without becoming quantitative research, coding could include useful indicators such as:

• number of participants contributing
• frequency
• discussion length
• intensity
• confidence
• centrality within theme networks

These measures should support, not replace, interpretation.

10. Standardised codebooks

Many qualitative researchers invent completely new coding systems. Some disciplines might benefit from validated codebooks that are adaptable, extensible, openly published, and comparable across studies. Similar developments occurred decades ago in psychology.

A possible future: "procedural thematic analysis"

Imagine if thematic analysis became as procedural as laboratory protocols. Instead of: Generate codes. The protocol might specify:

Step 1: Segment transcript into meaning units.
↓
Step 2: Assign provisional descriptive labels.
↓
Step 3: Compare against existing code definitions.
↓
Step 4: If similarity > threshold, use existing code.
↓
Otherwise create provisional code.
↓
Step 5: Review after five transcripts.
↓
Merge codes meeting defined similarity criteria.
↓
Recode earlier transcripts.
↓
Repeat until stopping rule reached.

Now two researchers would be much more likely to arrive at comparable analytical structures—not identical, but demonstrably following the same process.

Opportunities for doctoral innovation

This is a particularly fertile area for PhD research because it lies at the intersection of qualitative methodology, research quality, and emerging computational tools. Potential contributions include:

• Formalising coding protocols with explicit decision rules and stopping criteria.
• Developing quality metrics for codes and themes, rather than relying solely on researcher judgement.
• Designing visual analytic workflows that document how codes evolve into themes and, ultimately, into theoretical propositions.
• Integrating AI as an auditable assistant, where every AI suggestion is logged, accepted, modified, or rejected with reasons.
• Evaluating procedural consistency, by comparing how different analysts apply the same structured protocol versus traditional open-ended thematic analysis.

Such innovations would not remove the interpretive nature of qualitative research. Rather, they would make the process more transparent, more teachable, and more defensible. In the same way that statistical analysis is both interpretive and highly procedural, qualitative coding could become substantially more systematic without sacrificing the richness of human interpretation. For doctoral researchers, developing methods that increase transparency and analytical rigour while preserving interpretive depth represents one of the most promising avenues for advancing qualitative research methodology.