Bird Species Sound Classifier from Field Recordings

An audio machine-learning project that takes a field recording of a bird and suggests the likely species — with a confidence score and an explicit reminder that a human makes the final call. Recordings are converted into mel-spectrogram images, a convolutional classifier is trained on labeled community recordings from Xeno-canto, and predictions are presented as candidates for verification, not verdicts. It is the portfolio’s first audio-ML pipeline: waveforms in, spectrograms in between, species candidates out, and an honest uncertainty story throughout.

>An identification aid, not an authority. Predictions can be wrong; a single clip can contain multiple birds; noisy environments quietly degrade reliability; and some species are simply hard to separate by sound. Every prediction is a candidate to check against a field guide, a regional reference, or an expert birder — never a confirmed identification.

Who Is This For?

• Birders who want to understand how the apps they use actually work — by building a small one
• Bioacoustics students looking for a complete, reproducible audio-ML pipeline on real community data
• Citizen scientists who record birds and want to turn their archives into a labeled, searchable dataset
• ML learners who want a second modality (sound) beyond the usual image and tabular projects

The Problem

Birding by ear is a superpower that takes years to build, and the demand for it is enormous: migration seasons generate floods of “what was that?” moments, and automated recorders collect thousands of hours of audio that nobody has time to listen to. Commercial apps exist, but they are closed: you cannot inspect their training data, their error patterns, or their confidence behavior. Community-built databases like Xeno-canto host a genuinely open collection of labeled bird recordings from around the world — but turning raw community audio into a working classifier is the hard part, and it is exactly the part tutorials skip. The gap between “there is a big open dataset” and “I have a working, honest species-suggestion tool” is this project.

How It Works

Audio ML has its own shape. The signal is a one-dimensional waveform; the classic move is to transform it into a two-dimensional image-like representation (a spectrogram) and let image-style classifiers do the rest. Each stage below has a landmine; the pipeline text flags them where they bite.

1. Collect Recordings from Xeno-canto

Use the Xeno-canto API to fetch labeled recordings for a chosen set of species, along with their metadata (species, location, recordist, license). Start with a modest, geographically coherent species list rather than “all birds” — a focused regional subset trains better and evaluates more honestly. Record the query and retrieval date: community datasets grow and change, and provenance is part of the science. Respect the individual recording licenses when you redistribute anything.

2. Clean and Normalize the Audio

Field recordings arrive at different sampling rates, loudness levels, and channel counts. Standardize the sampling rate, convert to mono, and normalize amplitude. This is also where you trim silence at clip edges and decide how to handle clipped (over-loud) audio — models learn bad habits from distorted input.

3. Segment Longer Recordings

Many Xeno-canto recordings are longer than a training clip should be. Segment them into fixed-length windows (a few seconds each) and decide what to do with segments that contain no bird sound at all. Naive segmentation creates silent or noise-only training examples; even a simple energy threshold to discard near-silent windows pays for itself in model quality.

4. Convert to Mel-Spectrograms

Transform each segment into a mel-spectrogram — a time–frequency image with perceptually spaced frequency bins. The mel-spectrogram is the workhorse representation for environmental sound: it compresses the parts of the spectrum that matter for species discrimination while keeping the computation tractable. Keep the spectrogram parameters (window size, hop length, number of mel bands) consistent across the entire dataset; changing them mid-project silently invalidates your trained models.

5. Handle the Hard Realities

Three field-recording facts shape everything downstream:

• Background noise — wind, traffic, other animals; augmentation with light background noise makes models sturdier
• Overlapping calls — several birds may vocalize at once; a segment may legitimately contain more than one species, so frame the model’s job as suggesting the most prominent species
• Class imbalance — some species have thousands of recordings, others dozens; use balanced sampling or class-weighted loss rather than pretending the imbalance is not there

6. Train the Classifier

Fine-tune an image-style CNN on the mel-spectrograms. Because spectrograms are images, the transfer-learning machinery from vision projects carries over — but the “images” are spectrograms, so augmentations must be audio-legal: time-shifting and light noise injection yes, vertical flips no (upside-down frequencies are meaningless). Log per-class validation metrics throughout; audio models degrade quietly on rare classes long before aggregate metrics notice.

7. Predict and Surface Uncertainty

For a new clip, output the top-3 species candidates with probabilities. A high confidence on a clear single-bird clip means one thing; a low confidence on a noisy multi-bird dawn chorus means another — and the UI must say so. Confidence thresholds matter: below a chosen level, the tool should say “likely one of these, but verify” instead of pretending certainty.

8. Close the Loop with Human Verification

The finished tool always ends with the human: show the candidates, link to reference audio for each candidate species, and invite verification. Keeping a small log of user corrections is both good product design and a future training set.

Key Features

• Clip → species candidates with top-3 predictions and confidence scores
• Mel-spectrogram pipeline with consistent, documented parameters
• Xeno-canto-backed training data with recorded provenance and license respect
• Noise-aware augmentation for sturdier models on real field audio
• Class-imbalance handling via balanced sampling or class-weighted loss
• Verification loop — candidates link to reference recordings; corrections are logged

Functional Requirements

• Fetch recordings and metadata for a configured species list via the Xeno-canto API
• Standardize audio (sampling rate, mono, amplitude normalization) and discard near-silent windows
• Segment recordings into fixed-length windows and compute mel-spectrograms
• Train a fine-tuned CNN classifier with class-imbalance handling and per-class validation metrics
• Serve predictions for uploaded or recorded clips: top-3 species + confidence + “verify” guidance
• Export a model card listing species coverage, training-data date range, and measured per-class performance

User Stories

• As a birder, I want candidates with confidence scores so I know which clips deserve careful manual checking.
• As a bioacoustics student, I want the retrieval-to-training pipeline scripted so I can reproduce the dataset build.
• As a citizen scientist, I want to know which of my recordings the model is least sure about, so I can review those first.
• As a developer, I want per-class metrics so I can spot species that need more training data.
• As an educator, I want a small default species set so the whole pipeline trains on a laptop.

MVP Scope

A minimal but complete MVP:

Explicitly out of MVP: continuous recording analysis, on-device inference, many-hundred-species coverage, and call-type separation. All are natural extensions.

Project Timeline

• Week 1: API retrieval, audio cleaning, segmentation, spectrogram pipeline
• Week 2: baseline model, class-imbalance handling, first honest evaluation
• Week 3: augmentation experiments, per-class analysis, threshold behavior
• Week 4: verification UI, corrections log, model card, polish

Testing Strategy

• Data tests: every segment has a label and a species; spectrogram parameters identical across splits; no segment from the same original recording appears in both train and validation sets (recording-level splits prevent leakage)
• Model tests: metrics reproduce under a fixed seed; per-class metrics reported for every species in the set
• Robustness tests: predictions on lightly noise-augmented copies of validation clips degrade gracefully, not catastrophically
• UI tests: a clear single-species clip returns that species at high confidence; a pure-noise clip returns low confidence rather than a confident wrong answer

Security and Privacy Considerations

• Location metadata is sensitive. Xeno-canto metadata includes recording locations; some birds are rare or endangered, and precise locations can enable disturbance or poaching. Do not expose precise coordinates in your app’s outputs or demo datasets — work with species-level data in the UI and keep coordinates only in the training store, or strip them where the license and research purpose allow.
• Respect recording licenses: each Xeno-canto recording carries its own license terms; attribute recordists and do not redistribute beyond what the licenses permit.
• If you later add user-uploaded recordings, state clearly what is stored and for how long, and avoid building features that surveil specific places or people.
• Keep the model card explicit about coverage — a model trained on European species will confidently mislead on a South American forest.

Success Metrics

• A trained model with measured, reproducible per-class metrics on a held-out, recording-level split
• The full retrieval-to-training pipeline runs end to end from a clean checkout
• The prediction UI presents top-3 candidates, confidence, and verification guidance for every clip
• The model card documents species coverage, data provenance date, and known weaknesses (noisiest classes, most-confused pairs)
• User corrections are captured in a structured log

Common Challenges

• Noisy, heterogeneous field audio — real recordings are wind-battered and clipped; augmentation and honest robustness testing are not optional
• Overlapping species in one clip — the model suggests the most prominent species; multi-label classification is the principled extension
• Class imbalance — abundant species dominate; without balanced sampling, rare-species performance quietly collapses
• Recording-level leakage — segments from one recording spread across train and test inflate metrics; split by recording, not by segment
• Geographic bias — the training set reflects where recordists recorded, not where birds live; document coverage honestly
• Confidence calibration — softmax probabilities on spectrograms are often overconfident; show top-3 and verification guidance rather than a single verdict

Learning Objectives

• Build a complete audio-ML pipeline: retrieval, cleaning, segmentation, spectrogram conversion, classification
• Understand mel-spectrograms and why time–frequency representations suit environmental sound
• Handle the three real-world audio realities: noise, overlap, and class imbalance
• Practice recording-level splitting to avoid leakage in time-derived data
• Design a human-in-the-loop verification flow that treats the model as an aid, not an oracle

Why This Idea Is Different

The site’s machine-learning projects classify images (leaves, satellite patches, faces) and time series (sensor data, sales) — this is the first that classifies sound. The Crop Disease Detection with Computer Vision project fine-tunes a CNN on leaf photos; #057 fine-tunes the same family of model on mel-spectrogram images of sound, which is a genuinely different feature pipeline (waveform → time–frequency representation → CNN) for a different sensory world. There is no other audio-ML Idea on the site: no overlapping subject, no shared pipeline, and a distinct audience of birders and bioacoustics students rather than farmers or developers building dashboards.

What Similar Tools Exist

| Tool type | Approach | Limitation |
|———–|———-|————|
| Commercial bird-ID apps | Closed models on proprietary data | Uninspectable; error patterns and confidence behavior invisible |
| Research bioacoustic models | Large models from academic labs | Not student-buildable or rebuildable end to end |
| Generic audio classifiers | Pre-trained audio tags applied to clips | No species-specific training, no regional awareness |
| Expert human listening | Skilled birders identify by ear | Years of training; not scalable to recording archives |

This project’s differentiators: a fully open data-to-model pipeline on community recordings, per-class honesty about noisy and rare species, recording-level evaluation hygiene, and a verification-first product stance.

Technology Stack

• Python — the whole pipeline
• librosa / torchaudio — audio loading, resampling, mel-spectrograms
• PyTorch or TensorFlow/Keras — classifier training (image-style CNN over spectrograms)
• Pre-trained backbone (ResNet-family or a lightweight audio model) — transfer learning
• pandas + matplotlib + scikit-learn — metadata handling, metrics, charts
• Streamlit or a small web UI — prediction and verification interface

Future Enhancements

• Multi-label outputs for clips containing several species
• Region-aware models trained on biogeographic subsets
• On-device inference for offline field use
• Passive-acoustic-monitoring mode: scan long recorder files and timestamp detected vocalizations
• Active learning: surface low-confidence clips for expert labeling and retrain

Related Ideas

• Crop Disease Detection with Computer Vision — the shared transfer-learning backbone, applied to leaves instead of sound
• Satellite Land-Cover Explorer with EuroSAT Classification — the batch’s other new-modality classifier; spectrograms of terrain instead of sound
• Personalized Learning Path Generator — another learner-facing ML product built around user-specific adaptation

Browse more Project Ideas · Intermediate Ideas