Show HN: Data Engineering Book – An open source, community-driven guide
Hi HN! I'm currently a Master's student at USTC (University of Science and Technology of China). I've been diving deep into Data Engineering, especially in the context of Large Language Models (LLMs).
The Problem: I found that learning resources for modern data engineering are often fragmented and scattered across hundreds of medium articles or disjointed tutorials. It's hard to piece everything together into a coherent system.
The Solution: I decided to open-source my learning notes and build them into a structured book. My goal is to help developers fast-track their learning curve.
Key Features:
LLM-Centric: Focuses on data pipelines specifically designed for LLM training and RAG systems.
Scenario-Based: Instead of just listing tools, I compare different methods/architectures based on specific business scenarios (e.g., "When to use Vector DB vs. Keyword Search").
Hands-on Projects: Includes full code for real-world implementations, not just "Hello World" examples.
This is a work in progress, and I'm treating it as "Book-as-Code". I would love to hear your feedback on the roadmap or any "anti-patterns" I might have included!
Check it out:
Online: https://datascale-ai.github.io/data_engineering_book/
GitHub: https://github.com/datascale-ai/data_engineering_book
Show HN: Skill that lets Claude Code/Codex spin up VMs and GPUs
I've been working on CloudRouter, a skill + CLI that gives coding agents like Claude Code and Codex the ability to start cloud VMs and GPUs.
When an agent writes code, it usually needs to start a dev server, run tests, open a browser to verify its work. Today that all happens on your local machine. This works fine for a single task, but the agent is sharing your computer: your ports, RAM, screen. If you run multiple agents in parallel, it gets a bit chaotic. Docker helps with isolation, but it still uses your machine's resources, and doesn't give the agent a browser, a desktop, or a GPU to close the loop properly. The agent could handle all of this on its own if it had a primitive for starting VMs.
CloudRouter is that primitive — a skill that gives the agent its own machines. The agent can start a VM from your local project directory, upload the project files, run commands on the VM, and tear it down when it's done. If it needs a GPU, it can request one.
cloudrouter start ./my-project
cloudrouter start --gpu B200 ./my-project
cloudrouter ssh cr_abc123 "npm install && npm run dev"
Every VM comes with a VNC desktop, VS Code, and Jupyter Lab, all behind auth-protected URLs. When the agent is doing browser automation on the VM, you can open the VNC URL and watch it in real time. CloudRouter wraps agent-browser [1] for browser automation. cloudrouter browser open cr_abc123 "http://localhost:3000"
cloudrouter browser snapshot -i cr_abc123
# → @e1 [link] Home @e2 [link] Settings @e3 [button] Sign Out
cloudrouter browser click cr_abc123 @e2
cloudrouter browser screenshot cr_abc123 result.png
Here's a short demo: https://youtu.be/SCkkzxKBcPEWhat surprised me is how this inverted my workflow. Most cloud dev tooling starts from cloud (background agents, remote SSH, etc) to local for testing. But CloudRouter keeps your agents local and pushes the agent's work to the cloud. The agent does the same things it would do locally — running dev servers, operating browsers — but now on a VM. As I stopped watching agents work and worrying about local constraints, I started to run more tasks in parallel.
The GPU side is the part I'm most curious to see develop. Today if you want a coding agent to help with anything involving training or inference, there's a manual step where you go provision a machine. With CloudRouter the agent can just spin up a GPU sandbox, run the workload, and clean it up when it's done. Some of my friends have been using it to have agents run small experiments in parallel, but my ears are open to other use cases.
Would love your feedback and ideas. CloudRouter lives under packages/cloudrouter of our monorepo https://github.com/manaflow-ai/manaflow.
[1] https://github.com/vercel-labs/agent-browser
Show HN: Moltis – AI assistant with memory, tools, and self-extending skills
Hey HN. I'm Fabien, principal engineer, 25 years shipping production systems (Ruby, Swift, now Rust). I built Moltis because I wanted an AI assistant I could run myself, trust end to end, and make extensible in the Rust way using traits and the type system. It shares some ideas with OpenClaw (same memory approach, Pi-inspired self-extension) but is Rust-native from the ground up. The agent can create its own skills at runtime.
Moltis is one Rust binary, 150k lines, ~60MB, web UI included. No Node, no Python, no runtime deps. Multi-provider LLM routing (OpenAI, local GGUF/MLX, Hugging Face), sandboxed execution (Docker/Podman/Apple Containers), hybrid vector + full-text memory, MCP tool servers with auto-restart, and multi-channel (web, Telegram, API) with shared context. MIT licensed. No telemetry phoning home, but full observability built in (OpenTelemetry, Prometheus).
I've included 1-click deploys on DigitalOcean and Fly.io, but since a Docker image is provided you can easily run it on your own servers as well. I've written before about owning your content (https://pen.so/2020/11/07/own-your-content/) and owning your email (https://pen.so/2020/12/10/own-your-email/). Same logic here: if something touches your files, credentials, and daily workflow, you should be able to inspect it, audit it, and fork it if the project changes direction.
It's alpha. I use it daily and I'm shipping because it's useful, not because it's done.
Longer architecture deep-dive: https://pen.so/2026/02/12/moltis-a-personal-ai-assistant-bui...
Happy to discuss the Rust architecture, security model, or local LLM setup. Would love feedback.
Show HN: Skybolt Reflect – C++ header-only runtime reflection library
Skybolt-Reflect is an open-source JavaScript library that facilitates the creation of reflective interfaces, allowing developers to introspect and manipulate JavaScript objects at runtime. The library provides a set of APIs for accessing and modifying object properties, methods, and metadata, enabling advanced metaprogramming techniques.
Show HN: OpenWhisper – free, local, and private voice-to-text macOS app
I wanted a voice-to-text app but didn't trust any of the proprietary ones with my privacy.
So I decided to see if I could vibe code it with 0 macOS app & Swift experience.
It uses a local binary of whisper.cpp (a fast implementation of OpenAI's Whisper voice-to-text model in C++).
Github: https://github.com/richardwu/openwhisper
I also decided to take this as an opportunity to compare 3 agentic coding harnesses:
Cursor w/ Opus 4.6: - Best one-shot UI by far - Didn't get permissioning correct - Had issues making the "Cancel recording" hotkey being turned on all the time
Claude Code w/ Opus 4.6: - Fewest turns to get main functionality right (recording, hotkeys, permissions) - Was able to get a decent UI with a few more turns
Codex App w/ Codex 5.3 Extra-High: - Worst one-shot UI - None of the functionality worked without multiple subsequent prompts
Show HN: ClipPath – Paste screenshots as file paths in your terminal
ClipPath is an open-source library that provides a simple and efficient way to implement clipping paths in web applications. It offers cross-browser compatibility and supports various image formats, making it a useful tool for web developers working with complex visual elements.
Show HN: Geo Racers – Race from London to Tokyo on a single bus pass
Geo Racers is a mobile game that combines geography and racing, allowing players to explore real-world locations and compete in fast-paced races. The game aims to make learning about different countries and landmarks engaging and fun.
Show HN: Explore ASN Relationships and BGP Route History with Real Internet Data
Hi HN,
I’ve been working on a side project called ipiphistory.com.
It’s a searchable explorer for:
– ASN relationships (provider / peer / customer) – BGP route history – IP to ASN mapping over time – AS path visibility – Organization and geolocation data
The idea started from my frustration when explaining BGP concepts to junior engineers and students — most tools are fragmented across multiple sources (RouteViews, RIPE RIS, PeeringDB, etc.).
This project aggregates and indexes historical routing data to make it easier to:
– Understand how ASNs connect – Explore real-world routing behavior – Investigate possible hijacks or path changes – Learn BGP using real data
It’s still early and I’d really appreciate feedback from the HN community — especially on usability and features you’d like to see.
Happy to answer technical questions about data ingestion and indexing as well.
Show HN: Koala Diff – High-performance local data comparison (Rust and Polars)
The article describes Koala-Diff, a tool that efficiently compares and merges large binary files, reducing the time and resources required for version control and collaboration on binary data projects.
Show HN: ClawProxy: An HTTP proxy that injects auth tokens into API calls
A way to make you auth tokens totally hidden from OpenClaw. The idea:
* Put all auth tokens into a secrets directory
* Run OpenClaw in sandbox-exec mode using a shell wrapper. OpenClaw process is blocked by the OS from accessing secrets.
* OpenClaw routes API requests to HTTP proxy that injects auth tokens.
Show HN: Kuro-Nuri – Browser-based image redaction and compression using WASM
Hi HN,
I'm a beginner developer from Japan.
I built this tool because I was tired of opening Photoshop just to redact a name or a face from a screenshot. I didn't want to use existing "free online tools" because uploading sensitive images to a random server felt unsafe.
So I built Kuro-Nuri ("Blacked Out" in Japanese). It runs entirely in the browser using WebAssembly and TensorFlow.js (for auto-detecting faces). No data leaves your device.
Features: - Drag & drop to auto-redact faces. - Client-side compression. - Removes Exif metadata automatically.
The code is still a bit messy as I'm learning, but I'd love to hear your feedback on the performance and usability.
Thanks!
Show HN: Bubble Sort on a Turing Machine
Bubble sort is pretty simple in most programming languages ... what about on a Turing Machine? I used all three of Claude 4.6, GLM 5, and GPT 5.2 to get a result, so this exercise was not quite trivial, at least at this time. The resulting machine, bubble_sort_unary.yaml, will take this input:
111011011111110101111101111
and give this output:
101101110111101111101111111
I.e., it's sorting the array [3,2,7,1,5,4]. The machine has 31 states and requires 1424 steps before it comes to a halt. It also introduces two extra symbols onto the tape, 'A' and 'B'. (You could argue that 0 is also an extra symbol because turinmachine.io uses blank, ' ', as well).
When I started writing the code the LLM (Claude) balked at using unary numbers and so we implemented bubble_sort.yaml which uses the tape symbols '1', '2', '3', '4', '5', '6', '7'. This machine has fewer states, 25, and requires only 63 steps to perform the sort. So it's easier to watch it work, though it's not as generalized as the other TM.
Some comments about how the 31 states of bubbles_sort_unary.yaml operate:
| Group | Count | Purpose | |---|---|---| | `seek_delim_{clean,dirty}` | 2 | Pass entry: scan right to the next `0` delimiter between adjacent numbers. | | `cmpR_`, `cmpL_`, `cmpL_ret_`, `cmpL_fwd_` | 8 | Comparison: alternately mark units in the right (`B`) and left (`A`) numbers to compare their sizes. | | `chk_excess_`, `scan_excess_`, `mark_all_X_` | 6 | Excess check: right number exhausted — see if unmarked `1`s remain on the left (meaning L > R, swap needed). | | `swap_` | 7 | Swap: bubble each `X`-marked excess unit rightward across the `0` delimiter. | | `restore_*` | 6 | Restore: convert `A`, `B`, `X` marks back to `1`s, then advance to the next pair. | | `rewind` / `done` | 2 | Rewind to start after a dirty pass, or halt. |
(The above is in the README.md if it doesn't render on HN.)
I'm curious if anyone can suggest refinements or further ideas. And please send pull requests if you're so inclined. My development path: I started by writing a pretty simple INITIAL_IDEAS.md, which got updated somewhat, then the LLM created a SPECIFICATION.md. For the bubble_sort_unary.yaml TM I had to get the LLMs to build a SPEC_UNARY.md because too much context was confusing them. I made 21 commits throughout the project and worked for about 6 hours (I was able to multi-task, so it wasn't 6 hours of hard effort). I spent about $14 on tokens via Zed and asked some questions via t3.chat ($8/month plan).
A final question: What open source license is good for these types of mini-projects? I took the path of least resistance and used MIT, but I observe that turingmachine.io uses BSD 3-Clause. I've heard of "MIT with Commons Clause;" what's the landscape surrounding these kind of license questions nowadays?
Show HN: Holywell – The missing SQL formatter for sqlstyle.guide
For nearly a decade, I've wanted a SQL formatter that followed Simon Holywell's style guide (https://sqlstyle.guide). If you're unfamiliar with it, one of its key (and most controversial) features is “river alignment” of keywords. I couldn’t find an existing formatter that implemented it, so I made Holywell.
Try it in the browser: https://holywell.sh Repo: https://github.com/vinsidious/holywell
The site has a bunch of scrollable examples so you can quickly see what the formatted SQL looks like.
Dialect support is pretty basic right now (I’m mostly a Postgres user), but I’d love requests / failing examples for other dialects. Also, PRs are very welcome.
Disclaimer: not endorsed by Simon Holywell. I tried to be faithful to the guide (and where the guide is ambiguous, I had to interpret). Also: I’m not claiming this style is “best” — just that it’s the one I’ve wanted for a long time.
Please share your thoughts and let me know where it falls short!
Show HN: A reputation index from mitchellh's Vouch trust files
VouchBook is a decentralized platform that enables users to create, share, and verify personal recommendations and endorsements using blockchain technology. The platform aims to build trust and transparency in online interactions by leveraging the immutable nature of the blockchain.
Show HN: Ghost – Session memory for Claude Code (local, qmd, Git-integrated)
If you’ve used Claude Code on anything non-trivial, you’ve hit the wall. The project gets big enough that context falls apart between sessions. You start a new chat, re-explain the architecture, and watch it make the same mistake it made last week. Every session starts with this painful bootstrap where you’re trying to get the model back to where it was yesterday.
This obviously assumes Claude Code is doing most of the heavy lifting on your codebase. If you’re only using it for the occasional function, you probably don’t need this.
I spent a few days hacking on workarounds for this and eventually pulled them together into Ghost. It hooks into Claude Code sessions, summarises them, and indexes everything into QMD https://github.com/tobi/qmd for semantic search.
Next session, relevant context gets injected automatically. What you were working on, what decisions were made, what already failed.
It also keeps a mistake ledger. Things that went wrong get tracked and surfaced as warnings so you stop walking into the same walls.
Sessions are stored as markdown in .ai-sessions/ (gitignored).
Summaries get attached to commits as git notes so context travels with the code. Everything runs locally, nothing leaves your machine. Built with Bun. Hooks run under 100ms.
It’s early and rough but anecdotally it feels like it actually works.
Show HN: Sol LeWitt-style instruction-based drawings in the browser
Sol LeWitt was a conceptual artist who never touched his own walls.
He wrote instructions and other people executed them, the original prompt engineer!
I bookmarked a project called "Solving Sol" seven years ago and made a repo in 2018. Committed a README. Never pushed anything else.
Fast forward to 2026, I finally built it.
https://intervolz.com/sollewitt/
Show HN: AI agents play SimCity through a REST API
This is a weekend project that spiraled out of control. I was originally trying to get Claude to play a ROM of the SNES SimCity. I struggled with it and that led me to Micropolis (the open-sourced SimCity engine) and was able to get it to work by bolting on an API.
The weekend hack turned into a headless city simulation platform where anyone can get an API key (no signup) and have their AI agent play mayor. The simulation runs the real Micropolis engine inside Cloudflare Durable Objects, one per city. Every city is public and browsable on the site.
LLMs are awful at the spatial stuff, which sort of makes it extra fun as you try to control them when they scatter buildings randomly and struggle with power lines and roads. A little like dealing with a toddler.
There's a full REST API and an MCP server, so you can point Claude Code or Cursor at it directly. You can usually get agents building in seconds.
Website: https://hallucinatingsplines.com
API docs: https://hallucinatingsplines.com/docs
GitHub: https://github.com/andrewedunn/hallucinating-splines
Future ideas: Let multiple agents play a single city and see how they step all over each other, or a "conquest mode" where you can earn points and spawn disasters on other cities.
Show HN: I speak 5 languages. Common apps taught me none. So I built lairner
I'm Tim. I speak German, English, French, Turkish, and Chinese.
I learned Turkish with lairner itself -- after I built it. That's the best proof I can give you that this thing actually works.
The other four I learned the hard way: talking to people, making mistakes, reading things I actually cared about, and being surrounded by the language until my brain gave in. Every language app I tried got the same thing wrong: they teach you to pass exercises, not to speak. You finish a lesson, you get your dopamine hit, you maintain your streak, and six months later you still can't order food in the language you've been "learning."
So I built something different. lairner has 700+ courses across 70+ languages, including ones that Duolingo will never touch because there's no profit in it. Endangered languages. Minority languages. A Turkish speaker can learn Basque. A Chinese speaker can learn Welsh. Most platforms only let you learn from English. lairner lets you learn from whatever you already speak.
We work together with some institutes of endangered languages to be able to teach them on our platform.
It's a side project. I work a full-time dev job and build this in evenings and weekends. Tens of Thousands of users so far, no ad spend, no funding.
I'm not going to pretend this replaces living in a country or having a conversation partner. But I wanted something that at least tries to teach you the language instead of teaching you to play a language-themed game.
Happy to answer anything.
Show HN: A private, bulk audio converter using WASM (186x real-time speed)
The article describes a bulk audio converter tool that can remove vocals from music tracks, allowing users to isolate instrumental versions. The tool supports various audio formats and can process multiple files simultaneously.
Show HN: My agent started its own online store
I built Clawver (beta), infrastructure for AI agents to generate reliable income and run an online business end-to-end.
Agents can handle listing, checkout, fulfillment, and post-purchase flows via API (digital + POD), with Stripe payouts and webhooks for automation. Minimal human intervention, only where required (Stripe onboarding).
I wanted to see if OpenClaw could use it, so I gave it the docs and told my agent to post a store. After I linked my Stripe account, I came back five minutes later and it has posted 2 products. Crazy what's possible now with a smart agent and API access.
Check it out at https://clawver.store . Feel free to build your own agent and lmk what you think.
Show HN: CodeRLM – Tree-sitter-backed code indexing for LLM agents
I've been building a tool that changes how LLM coding agents explore codebases, and I wanted to share it along with some early observations.
Typically claude code globs directories, greps for patterns, and reads files with minimal guidance. It works in kind of the same way you'd learn to navigate a city by walking every street. You'll eventually build a mental map, but claude never does - at least not any that persists across different contexts.
The Recursive Language Models paper from Zhang, Kraska, and Khattab at MIT CSAIL introduced a cleaner framing. Instead of cramming everything into context, the model gets a searchable environment. The model can then query just for what it needs and can drill deeper where needed.
coderlm is my implementation of that idea for codebases. A Rust server indexes a project with tree-sitter, builds a symbol table with cross-references, and exposes an API. The agent queries for structure, symbols, implementations, callers, and grep results — getting back exactly the code it needs instead of scanning for it.
The agent workflow looks like:
1. `init` — register the project, get the top-level structure
2. `structure` — drill into specific directories
3. `search` — find symbols by name across the codebase
4. `impl` — retrieve the exact source of a function or class
5. `callers` — find everything that calls a given symbol
6. `grep` — fall back to text search when you need it
This replaces the glob/grep/read cycle with index-backed lookups. The server currently supports Rust, Python, TypeScript, JavaScript, and Go for symbol parsing, though all file types show up in the tree and are searchable via grep.
It ships as a Claude Code plugin with hooks that guide the agent to use indexed lookups instead of native file tools, plus a Python CLI wrapper with zero dependencies.
For anecdotal results, I ran the same prompt against a codebase to "explore and identify opportunities to clarify the existing structure".
Using coderlm, claude was able to generate a plan in about 3 minutes. The coderlm enabled instance found a genuine bug (duplicated code with identical names), orphaned code for cleanup, mismatched naming conventions crossing module boundaries, and overlapping vocabulary. These are all semantic issues which clearly benefit from the tree-sitter centric approach.
Using the native tools, claude was able to identify various file clutter in the root of the project, out of date references, and a migration timestamp collision. These findings are more consistent with methodical walks of the filesystem and took about 8 minutes to produce.
The indexed approach did better at catching semantic issues than native tools and had a key benefit in being faster to resolve.
I've spent some effort to streamline the installation process, but it isn't turnkey yet. You'll need the rust toolchain to build the server which runs as a separate process. Installing the plugin from a claude marketplace is possible, but the skill isn't being added to your .claude yet so there are some manual steps to just getting to a point where claude could use it.
Claude continues to demonstrate significant resistance to using CodeRLM in exploration tasks. Typically to use you will need to explicitly direct claude to use it.
---
Repo: github.com/JaredStewart/coderlm
Paper: Recursive Language Models https://arxiv.org/abs/2512.24601 — Zhang, Kraska, Khattab (MIT CSAIL, 2025)
Inspired by: https://github.com/brainqub3/claude_code_RLM
Show HN: Pgclaw – A "Clawdbot" in every row with 400 lines of Postgres SQL
Hi HN,
Been hacking on a simple way to run agents entirely inside of a Postgres database, "an agent per row".
Things you could build with this: * Your own agent orchestrator * A personal assistant with time travel * (more things I can't think of yet)
Not quite there yet but thought I'd share it in its current state.
Show HN: Rowboat – AI coworker that turns your work into a knowledge graph (OSS)
Hi HN,
AI agents that can run tools on your machine are powerful for knowledge work, but they’re only as useful as the context they have. Rowboat is an open-source, local-first app that turns your work into a living knowledge graph (stored as plain Markdown with backlinks) and uses it to accomplish tasks on your computer.
For example, you can say "Build me a deck about our next quarter roadmap." Rowboat pulls priorities and commitments from your graph, loads a presentation skill, and exports a PDF.
Our repo is https://github.com/rowboatlabs/rowboat, and there’s a demo video here: https://www.youtube.com/watch?v=5AWoGo-L16I
Rowboat has two parts:
(1) A living context graph: Rowboat connects to sources like Gmail and meeting notes like Granola and Fireflies, extracts decisions, commitments, deadlines, and relationships, and writes them locally as linked and editable Markdown files (Obsidian-style), organized around people, projects, and topics. As new conversations happen (including voice memos), related notes update automatically. If a deadline changes in a standup, it links back to the original commitment and updates it.
(2) A local assistant: On top of that graph, Rowboat includes an agent with local shell access and MCP support, so it can use your existing context to actually do work on your machine. It can act on demand or run scheduled background tasks. Example: “Prep me for my meeting with John and create a short voice brief.” It pulls relevant context from your graph and can generate an audio note via an MCP tool like ElevenLabs.
Why not just search transcripts? Passing gigabytes of email, docs, and calls directly to an AI agent is slow and lossy. And search only answers the questions you think to ask. A system that accumulates context over time can track decisions, commitments, and relationships across conversations, and surface patterns you didn't know to look for.
Rowboat is Apache-2.0 licensed, works with any LLM (including local ones), and stores all data locally as Markdown you can read, edit, or delete at any time.
Our previous startup was acquired by Coinbase, where part of my work involved graph neural networks. We're excited to be working with graph-based systems again. Work memory feels like the missing layer for agents.
We’d love to hear your thoughts and welcome contributions!
Show HN: Codex HUD – Claude-HUD Style Status Line for Codex CLI
I built Codex HUD to make Codex CLI sessions easier to monitor without leaving the terminal.
It adds a real-time status line with:
- active model
- project + git branch/dirty state
- 5h and 7d usage bars
- automatic Spark vs default limit selection
Quick install:
git clone https://github.com/anhannin/codex-hud.git
cd codex-hud/Codex-HUD
./install.sh
Feedback I’m looking for:
- portability across Linux distros/shell setups
- readability on narrow terminal widths
- edge cases in usage/rate parsing
Repo issues are welcome if you hit bugs.
Show HN: Agent Alcove – Claude, GPT, and Gemini debate across forums
Show HN: Toil, a go library for simple parallelism
I was tired of having to write the same basic primitive over and over again: A channel, some control logic, etc.
So I wrote toil -- A port of two of my favorite Python functions over into the Go world. It's very simple. There's optimizations to be made for sure, but this is the result of a couple of hours of wanting something that felt Go-Like in the right way.
Show HN: 20+ Claude Code agents coordinating on real work (open source)
Single-agent LLMs suck at long-running complex tasks.
We’ve open-sourced a multi-agent orchestrator that we’ve been using to handle long-running LLM tasks. We found that single LLM agents tend to stall, loop, or generate non-compiling code, so we built a harness for agents to coordinate over shared context while work is in progress.
How it works: 1. Orchestrator agent that manages task decomposition 2. Sub-agents for parallel work 3. Subscriptions to task state and progress 4. Real-time sharing of intermediate discoveries between agents
We tested this on a Putnam-level math problem, but the pattern generalizes to things like refactors, app builds, and long research. It’s packaged as a Claude Code skill and designed to be small, readable, and modifiable.
Use it, break it, tell me about what workloads we should try and run next!
Show HN: What is HN thinking? Real-time sentiment and concept analysis
Hi HN,
I made Ethos, an open-source tool to visualize the discourse on Hacker News. It extracts entities, tracks sentiment, and groups discussions by concept.
Check it out: https://ethos.devrupt.io
This was a "budget build" experiment. I managed to ship it for under $1 in infra costs. Originally I was using `qwen3-8b` for the LLM and `qwen3-embedding-8b` for the embedding, but I ran into some capacity issues with that model and decided to use `llama-3.1-8b-instruct` to stay within a similar budget while having higher throughput.
What LLM or embedding would you have used within the same price range? It would need to be a model that supports structured output.
How bad do you think it is that `llama-3.1` is being used and then a higher dimension embedding? I originally wanted to keep the LLM and embedding within the same family, but I'm not sure if there is munch point in that.
Repo: https://github.com/devrupt-io/ethos
I'm looking for feedback on which metrics (sentiment vs. concepts) you find most interesting! PRs welcome!
Show HN: I built a macOS tool for network engineers – it's called NetViews
Hi HN — I’m the developer of NetViews, a macOS utility I built because I wanted better visibility into what was actually happening on my wired and wireless networks.
I live in the CLI, but for discovery and ongoing monitoring, I kept bouncing between tools, terminals, and mental context switches. I wanted something faster and more visual, without losing technical depth — so I built a GUI that brings my favorite diagnostics together in one place.
About three months ago, I shared an early version here and got a ton of great feedback. I listened: a new name (it was PingStalker), a longer trial, and a lot of new features. Today I’m excited to share NetViews 2.3.
NetViews started because I wanted to know if something on the network was scanning my machine. Once I had that, I wanted quick access to core details—external IP, Wi-Fi data, and local topology. Then I wanted more: fast, reliable scans using ARP tables and ICMP.
As a Wi-Fi engineer, I couldn’t stop there. I kept adding ways to surface what’s actually going on behind the scenes.
Discovery & Scanning: * ARP, ICMP, mDNS, and DNS discovery to enumerate every device on your subnet (IP, MAC, vendor, open ports). * Fast scans using ARP tables first, then ICMP, to avoid the usual “nmap wait”.
Wireless Visibility: * Detailed Wi-Fi connection performance and signal data. * Visual and audible tools to quickly locate the access point you’re associated with.
Monitoring & Timelines: * Connection and ping timelines over 1, 2, 4, or 8 hours. * Continuous “live ping” monitoring to visualize latency spikes, packet loss, and reconnects.
Low-level Traffic (but only what matters): * Live capture of DHCP, ARP, 802.1X, LLDP/CDP, ICMP, and off-subnet chatter. * mDNS decoded into human-readable output (this took months of deep dives).
Under the hood, it’s written in Swift. It uses low-level BSD sockets for ICMP and ARP, Apple’s Network framework for interface enumeration, and selectively wraps existing command-line tools where they’re still the best option. The focus has been on speed and low overhead.
I’d love feedback from anyone who builds or uses network diagnostic tools: - Does this fill a gap you’ve personally hit on macOS? - Are there better approaches to scan speed or event visualization that you’ve used? - What diagnostics do you still find yourself dropping to the CLI for?
Details and screenshots: https://netviews.app There’s a free trial and paid licenses; I’m funding development directly rather than ads or subscriptions. Licenses include free upgrades.
Happy to answer any technical questions about the implementation, Swift APIs, or macOS permission model.
Show HN: Forkwatch – Discover meaningful patches hiding in GitHub forks
Hey HN,
I had Claude build a CLI tool that analyzes GitHub forks to surface changes that haven't been submitted as PRs.
The core idea is convergence: when multiple independent forks touch the same file and make the same change, that's a strong signal something needs fixing upstream.
Example: I ran forkwatch against a Ruby API client library and found 11 independent forks all upgrading the same stale dependency. 4 of them made byte-for-byte identical changes to another file.
$ forkwatch analyze maximadeka/convertkit-ruby
convertkit-ruby.gemspec (11 forks converge here)
WebinarGeek +1 -2 — Change gitspec faraday version
- spec.add_runtime_dependency "faraday", "~> 1.0"
- spec.add_runtime_dependency "faraday_middleware", "~> 1.0"
+ spec.add_runtime_dependency "faraday", '>= 2.0'
...
lib/convertkit/connection.rb (4 forks converge here) Most common change pattern:
require "faraday"
-require "faraday_middleware"
require "json"
WebinarGeek, chaiandconversation, alexbndk, excid3
It filters out noise (dependabot, lock files, CI config), groups forks by files changed, and deduplicates identical patches. There's a --json flag for scripting/AI and a --patch flag that outputs a unified diff you can pipe to git apply.Uses the GitHub CLI for auth and one API call per fork. Written in Go.
https://github.com/stympy/forkwatch