Detecting Overhanging Trees on Residential Roofs
Why Overhanging Trees Matter
Trees that extend over residential roofs are more than an aesthetic concern. They represent a significant and quantifiable risk factor for property damage. Overhanging branches drop leaves that clog gutters and trap moisture, accelerating roof deterioration. During storms, heavy limbs can break and puncture roofing materials. Root systems from trees growing close to structures can compromise foundations. And in wildfire-prone regions, overhanging vegetation creates a direct fuel pathway from ground-level fire to the roof.
For insurance companies, overhanging trees are a key underwriting variable. A property with significant tree overhang has a materially higher risk profile for wind, hail, and fire claims. Inspectors currently assess overhang through on-site visits or manual review of aerial photos—both expensive and difficult to scale across portfolios of millions of properties.
Automating overhanging tree detection from aerial imagery could transform how property risk is assessed, enabling portfolio-wide analysis at a fraction of the cost of manual inspection.
The Dataset
We worked with a dataset of 33 high-resolution aerial images annotated with approximately 21,677 polygons across three classes:
- Structures with roofs (~7,200 annotations): Residential buildings, garages, sheds, and other roofed structures. These polygons trace the visible roof footprint.
- Trees and shrubs (~11,500 annotations): Vegetation canopy polygons, including trees of all sizes, large shrubs, and hedge rows. The annotations follow the visible canopy boundary from the aerial perspective.
- Overhanging trees (~2,977 annotations): The intersection class—regions where tree or shrub canopy extends over a roof structure. These are computed geometrically by finding the overlap between roof polygons and vegetation polygons.
The geometric computation of overhang is straightforward in principle: a tree overhangs a roof if and only if the tree canopy polygon intersects the roof polygon. The overhang region is the geometric intersection of these two polygons. This gives us clean ground truth labels derived from the two base annotation classes.
In practice, the annotations reveal the complexity of the problem. Overhang regions range from tiny corner intrusions (a single branch tip crossing the roof edge) to extensive coverage (a large tree whose canopy spans most of a small structure). Some properties have multiple overhanging trees from different directions, creating complex overhang patterns.
Approach 1: UNet for Three-Class Segmentation
Our first approach trained a standard UNet to simultaneously segment all three classes: roofs, trees, and overhanging trees. The intuition was that the model could learn the spatial relationship between these classes jointly, using roof and tree context to inform overhang predictions.
Architecture: We used a UNet with a ResNet-34 encoder pretrained on ImageNet, three output channels (one per class), and pixel-wise cross-entropy loss. Images were tiled into 512x512 patches with overlap for inference.
Results: The model performed well on roofs and trees individually—these are relatively standard segmentation targets with clear visual features. However, the overhanging tree class proved much harder. The model struggled because overhang is not purely a visual concept: it is a spatial relationship between two other classes. Pixels in the overhang region look like tree canopy (because they are tree canopy), and the model had difficulty learning that these particular tree pixels are special because they happen to be above a roof.
The model achieved reasonable recall on large, obvious overhang regions but missed subtle cases where only a few branches crossed the roof boundary.
Approach 2: UNet for Overhang Only
Our second approach simplified the task: train a UNet to directly predict only the overhanging tree class, treating it as binary segmentation (overhang vs. everything else).
Rationale: By focusing the model’s capacity entirely on the overhang class, we hoped to improve detection of subtle overhang regions. The model would need to implicitly learn what roofs and trees look like (since overhang only occurs at their intersection) but would not be penalized for mistakes on non-overhang pixels.
Results: This model showed improved precision on the overhang class—fewer false positives in areas with trees near but not over roofs. However, recall remained limited for small overhang regions. The fundamental challenge persisted: the model needed to understand spatial relationships between classes, not just classify pixels based on local appearance.
Approach 3: Distance-Based Method
Recognizing that overhang is fundamentally about the spatial proximity of trees to roofs, our third approach took an explicitly geometric route.
Method: First, segment roofs and trees independently using two separate, well-performing UNet models. Then, compute the distance between predicted tree boundaries and predicted roof boundaries. Regions where tree canopy falls within a threshold distance of (or overlaps with) a roof polygon are classified as overhang.
Steps:
- Predict roof segmentation masks using a roof-specific UNet.
- Predict tree segmentation masks using a tree-specific UNet.
- Compute pairwise distances between tree and roof polygon boundaries.
- Identify overlap regions (distance = 0, meaning geometric intersection) as overhang.
- Optionally extend overhang to include near-overhang regions (distance < threshold) to capture branches that are close to but not yet over the roof.
Results: This approach produced more interpretable results because each step could be evaluated independently. When the roof and tree segmentation models performed well, the geometric intersection closely matched ground truth overhang annotations. The method was also more robust to small segmentation errors—a slightly imprecise tree boundary still produced a reasonable overhang estimate.
The main limitation was cascade error: mistakes in either the roof or tree segmentation propagated directly into overhang estimates. A missed roof corner or an over-segmented tree canopy would create false negatives or false positives in the overhang output.
Approach 4: Unsupervised Branch-Level Segmentation
Our fourth approach explored whether tree overhang could be detected without any overhang-specific training data, using unsupervised methods to segment individual tree branches.
Concept: If we can segment the fine structure of tree canopy—individual branches and limbs—we can identify which specific branches extend over roof structures. This branch-level analysis could provide more actionable information than blob-level overhang detection: not just “there is overhang” but “these specific branches are the problem.”
Method: We applied unsupervised segmentation techniques (including superpixel methods and graph-based segmentation) to high-resolution tree canopy regions, aiming to decompose the canopy into individual branch clusters. These clusters were then tested for spatial overlap with roof polygons.
Results: The unsupervised approach produced interesting qualitative results—it could identify branch structures within canopy regions that other methods treated as monolithic blobs. However, the segmentation was noisy and inconsistent. The definition of a “branch” in an aerial image is inherently ambiguous at most resolutions, and unsupervised methods produced variable granularity depending on image quality, tree species, and seasonal leaf cover.
This approach is promising for future work with higher-resolution imagery (sub-centimeter) where individual branches become more visually distinct, but it was not reliable enough for production use at the resolutions available in our dataset.
Approach 5: Multi-Season Tree State Labeling
Our final approach addressed a dimension that all previous methods ignored: temporal variation. Trees change dramatically across seasons. A deciduous tree that heavily overhangs a roof in summer (full canopy) may pose minimal risk in winter (bare branches). Conversely, the bare-branch winter view reveals the actual branch structure and proximity to the roof more clearly than the summer canopy view.
Method: We labeled tree state (leafed vs. bare vs. partially leafed) across imagery from different seasons for the same properties. The goal was to develop models that:
- Assess overhang risk from any seasonal condition.
- Combine multi-season observations for more robust estimates.
- Distinguish between “current overhang” (canopy currently over roof) and “structural overhang risk” (branches positioned to overhang when leafed).
Results: Multi-season analysis provided significantly richer information. Summer imagery best captured the full extent of canopy overhang, while winter imagery best revealed branch proximity and structural risk. Properties observed across seasons received more accurate risk assessments than those observed at only one time point.
The practical challenge was data availability: many properties are only imaged once per year, and aligning multi-season imagery for the same property introduces registration and normalization complications.
Comparative Assessment
| Approach | Strengths | Limitations |
|---|---|---|
| 3-class UNet | Single model, joint learning | Struggles with relational concept |
| Overhang-only UNet | Focused capacity | Low recall on subtle cases |
| Distance-based | Interpretable, modular | Cascade error propagation |
| Unsupervised branches | No overhang labels needed, fine-grained | Noisy, resolution-dependent |
| Multi-season | Richer risk assessment | Data availability constraints |
Key Takeaways
Overhang detection is a relationship problem, not just a classification problem. The pixel-level appearance of an overhanging tree branch is identical to a non-overhanging branch—the difference is entirely spatial context. This makes it fundamentally different from standard segmentation tasks and suggests that methods explicitly encoding spatial relationships (like the distance-based approach) have an inherent advantage over those that treat it as pure pixel classification.
Modular pipelines outperformed end-to-end models in our experiments. Training separate, high-quality models for roofs and trees, then combining their outputs geometrically, produced more reliable and interpretable results than trying to learn overhang directly from pixels.
Multi-season analysis is underexploited in aerial imagery applications. Most property assessment relies on single-date imagery, but the seasonal variation in tree cover contains signal that is lost in any single observation. As multi-temporal aerial imagery becomes more available, methods that fuse seasonal observations will become increasingly valuable.
The practical path forward likely combines the distance-based approach (for reliable production deployment today) with multi-season analysis (for improved accuracy as temporal data becomes more accessible), using unsupervised branch segmentation as a research direction for next-generation, high-resolution imagery analysis.