mirror of
https://github.com/thatmattlove/hyperglass.git
synced 2026-01-30 05:49:22 +00:00
hyperglass is the network looking glass that tries to make the internet better.
4a1057651f
MAJOR ENHANCEMENTS: IP Enrichment Service (hyperglass/external/ip_enrichment.py): - Increase IXP data cache duration from 24 hours to 7 days (604800s) for better performance - Fix critical cache refresh logic: ensure_data_loaded() now properly checks expiry before using existing pickle files - Remove 'force' refresh parameters from public APIs and admin endpoints to prevent potential abuse/DDOS - Implement automatic refresh based on file timestamps and cache duration - Add comprehensive debug logging gated by Settings.debug throughout the module - Clean up verbose comments and improve code readability - Update configuration model to enforce 7-day minimum cache timeout MikroTik Traceroute Processing: - Refactor trace_route_mikrotik plugin to use garbage cleaner before structured parsing - Only log raw router output when Settings.debug is enabled to reduce log verbosity - Simplify MikrotikTracerouteTable parser to expect pre-cleaned input from garbage cleaner - Remove complex multi-table detection, format detection, and deduplication logic (handled by cleaner) - Add concise debug messages for processing decisions and configuration states Traceroute IP Enrichment (traceroute_ip_enrichment.py): - Implement concurrent reverse DNS lookups using asyncio.to_thread and asyncio.gather - Add async wrapper for reverse DNS with proper error handling and fallbacks - Significant performance improvement for multi-hop traceroutes (parallel vs sequential DNS) - Proper debug logging gates: only detailed logs when Settings.debug=True - Upgrade operational messages to log.info level (start/completion status) - Maintain compatibility with different event loop contexts and runtime environments Configuration Updates: - Update structured.ip_enrichment.cache_timeout default to 604800 seconds - Update documentation to reflect new cache defaults and behavior - Remove force refresh options from admin API endpoints MIGRATION NOTES: - Operators should ensure /etc/hyperglass/ip_enrichment directory is writable - Any code relying on force refresh parameters must be updated - Monitor logs for automatic refresh behavior and performance improvements - The 7-day cache significantly reduces PeeringDB API load PERFORMANCE BENEFITS: - Faster traceroute enrichment due to concurrent DNS lookups - Reduced external API calls with longer IXP cache duration - More reliable refresh logic prevents stale cache usage - Cleaner, more focused debug output when debug mode is disabled TECHNICAL DETAILS: - Uses asyncio.to_thread for non-blocking DNS operations - Implements process-wide file locking for safe concurrent cache updates - Robust fallbacks for various asyncio execution contexts - Maintains backward compatibility while improving performance FILES MODIFIED: - hyperglass/external/ip_enrichment.py - hyperglass/models/config/structured.py - hyperglass/api/routes.py - hyperglass/plugins/_builtin/trace_route_mikrotik.py - hyperglass/models/parsing/mikrotik.py - hyperglass/plugins/_builtin/traceroute_ip_enrichment.py - docs/pages/configuration/config/structured-output.mdx |
||
|---|---|---|
| .github | ||
| .samples | ||
| .tests | ||
| .vscode | ||
| docs | ||
| hyperglass | ||
| .dockerignore | ||
| .gitattributes | ||
| .gitignore | ||
| biome.json | ||
| CHANGELOG.md | ||
| CODE_OF_CONDUCT.md | ||
| compose.yaml | ||
| CONTRIBUTING.md | ||
| debug_mikrotik_minimal.py | ||
| Dockerfile | ||
| example_community_names_config.yaml | ||
| hooks.sh | ||
| LICENSE | ||
| poetry.lock | ||
| pyproject.toml | ||
| README.md | ||
| requirements-dev.lock | ||
| requirements.lock | ||
| test_community_names.py | ||
| TRACEROUTE_RESTRUCTURE_SUMMARY.md | ||
| version.py | ||
The network looking glass that tries to make the internet better.
A looking glass is implemented by network operators as a way of providing customers, peers, or the general public with a way to easily view elements of, or run tests from the provider's network.
hyperglass is intended to make implementing a looking glass too easy not to do, with the lofty goal of improving the internet community at large by making looking glasses more common across autonomous systems of any size.
Changelog
Features
- BGP Route, BGP Community, BGP AS Path, Ping, & Traceroute, or add your own commands.
- Structured data output with rich metadata for supported platforms
- Enhanced traceroute with ASN information, organization names, country codes, and IXP detection
- Full IPv6 support
- Customizable everything: features, theme, UI/API text, error messages, commands
- Built-in support for:
- Arista EOS
- BIRD
- Cisco IOS
- Cisco NX-OS
- Cisco IOS-XR
- FRRouting
- Huawei VRP
- Juniper Junos
- Mikrotik
- Nokia SR OS
- OpenBGPD
- TNSR
- VyOS
- Structured BGP Route support for: Arista EOS, FRRouting, Huawei VRP, Juniper Junos, MikroTik RouterOS
- Structured Traceroute support for: Arista EOS, FRRouting, Huawei VRP, Juniper Junos, MikroTik RouterOS/SwitchOS
- Configurable support for any other supported platform
- Optionally access devices via an SSH proxy/jump server
- Access-list/prefix-list style query control to whitelist or blacklist query targets
- REST API with automatic, configurable OpenAPI documentation
- Modern, responsive UI built on ReactJS, with NextJS & Chakra UI, written in TypeScript
- AS path visualization with interactive flow charts showing organization names
- Offline IP enrichment using BGP.tools bulk data and PeeringDB for maximum performance
- Query multiple devices simultaneously
- Concurrent processing with non-blocking operations for improved performance
- Browser-based DNS-over-HTTPS resolution of FQDN queries
To request support for a specific platform, please submit a Github Issue with the feature label.
Get Started →
Community
Any users, potential users, or contributors of hyperglass are welcome to join and discuss usage, feature requests, bugs, and other things.
hyperglass is developed with the express intention of being free to the networking community.
However, if you're feeling particularly helpful or generous, small donations are welcome.
Acknowledgements
hyperglass is built entirely on open-source software. Here are some of the awesome libraries used, check them out too!
- Netmiko
- Litestar
- Pydantic
- Chakra UI
- React Flow - AS path visualization
- BGP.tools - IP enrichment data
- PeeringDB - Network organization and IXP data
