Spent last night migrating a Discord bot from Heroku to Fly as a result of the upcoming closure of the Heroku free dynos. Overall fairly painless, though I opted to provide my own images rather than use a source-to-image pipeline and discovered one little quirk: images require a Docker V2 formatted manifest. I use Buildah and Podman in my workflows which default to the OCI format. This was simple enough to solve once I figured it out, but I only found one or two forum posts on it and spent a lot of time trying to figure out why the deployment couldn't find an image that I manually pushed to registry.fly.io.

I absolutely love Fly. Cannot recommend them enough for the right use cases.

Fly made infrastructure really hard for me because their hypervisors cpuid emulation broke my program.

Yes, fly employees, I will file a bug somewhere - or email me.

Sadly, we have all seen these promises, "X makes Y much easier" but you cannot make complex things easy without removing lots of functionality.

Kubernetes for the basics is actually pretty easy despite what people say, I got a fairly simple cluster running with little pain but...it doesn't take long before you want multiple clusters, or vlan peering, or customising the DNS or.... and that is when it becomes complex.

What will fly.io do? Probably what everyone else does, starts simple, becomes popular and then caves in to the deluge of feature requests until they end up with an Azure/AWS/GCP clone. If it stays simple, lots of people will say that you will outgrow it quickly and need something else, if you increase functionality, you lose your USP of making infrastructure easy.

I think perhaps the abstractions are the problem, if you are abstracting at the same level as everyone else e.g. docker images, orchestration etc. then I don't understand how it can ever work differently.

To make my point, the very first comment below (above?) is about container format, a really fundamental thing that noobs are not likely to know about, they will just immediately have some kind of error.

It seems Fly.io originally focused on Elixir, but has identified the product-market need for app server + Postgres.

We didn't know about Fly.io when we chose GCP for this setup.

The initial setup on GCP was exceedingly painful. We used CloudRun for our app server, with the value prop being that "it just works". It didn't. Our container failed to start with zero logs from our servers. Stackdriver was of no help. Eventually we found a Stackoverflow thread revealing that CloudRun didn't like Docker images built from Macs. As always, GCP's official docs and resources are incoherent. GCP docs address a hundred things you don't care about, and the signal-to-noise percentage is in the low teens, if we're being generous. We had to chase down half a dozen bureaucratic things to get our CloudRun app to see and talk to CloudSQL. Apparently with Fly.io, you just run a command to provision Postgres, and pass in an environment variable to your app.

We consoled ourselves that GCP was difficult to setup but now it's set-and-forget. This is also a lie. This week we saw elevated and unexplained 5xx. First was CloudRun randomly disconnecting from CloudSQL. As AWS measures reliability in terms of 9s, the way GCP DevRel responded to this bug is that this is a distributed system and therefore acceptable that things just fail a reliably human-reproducable 1%+ error rate. Yesterday we saw botnet traffic scanning for vulnerabilities on our app. This happens if you're on the web, not inherently GCP's fault. We have GCP's Cloud Load balancer setup but it's not very smart. We were able to manually block specific IP addresses but it's no where as meaningful as Cloudflare. Not a fan of Cloudflare the company but their products address a need. The botnet somehow knocked over our "Serverless VPC connection" to CloudSQL. Basically what that is is a proxy server that you are forced to setup because CloudRun can't actually talk to cloudSQL. All the auto-scaling claims of GCP's serverless are diminished if we are forced to introduce a single point of failures like this in the loop. That serverless VPC connection requires a minimum of 2+ VMs, so the scale-to-zero of CloudRun is no more. Our experience with GCP is constantly having to come up with workarounds and addressing their accidental complexity. This should not be the customer's problem. For example, CloudSQL doesn't have an interface to query your databases. If you use a private IP for security, you can't even use GCP's command line tools to access this. We found out that GCE VMs are automatically networked to talk to CloudSQL. We ended up creating a "bastion" GCE VM instance to ad-hoc query our DB state. For this, we just needed to the cheapest VM but GCP makes even this difficult. As for Stackdriver, it's still been an annoyingly painful UI.

Does fly recommend any good tools for measuring client latency (which is their big selling point)? I know they offer graphs for server latency.

Without PITR for postges I don’t think so.

I'm an Elixir developer and I've used Fly.io a good bit for a project I've been working on. I'm also wicked smart, have 15 years of software development experience, and I'm looking for work. Drop me a note at the email address in my profile if you're interested in hiring me!

The thing is about all these tools is that starting out with a simple VPS is much more cost effective? https://contabo.com has 4 cores for 5€ and you can scale up to 10 cores and 60 gb ram for 27€ a month. And if you need dedicated cores 12 cores with 96GB is 130€ vs 8 dedicated cores with 64 GB ram for 550€

Choose a docker image and just docker-compose up your application.

If you outgrow that, you might aswell switch to kubernetes and aws/gcp/azure

I had a good experience doing the test app with fly, I’d definitely consider it for something internet routable and non-complex like a bot or something. Very excited for when I can run more complicated workloads such as workloads started with Docker Compose, without having to implement all the Docker Compose functionality myself.

I like Fly but can't migrate my apps to them because they don't support cron jobs - they've been promising it for more than a year now: https://community.fly.io/t/recurring-scheduled-tasks-like-cr...

There is nothing easy about them, cheaper yes (only for the lower instances), but not easier. Until they have first-class, evergreen, support for Heroku buildpacks, they are a subpar replacement.

They are also missing a proper managed database with point-in-time backups through the web UI like those offered by most proper PaaS services.

There are a bunch of rough edges.

Your server doesn't have a static IP for outgoing requests, so to use it with RDS, you can't just open up a port on the RDS side. (They want you to set up your own proxy) https://community.fly.io/t/set-get-outgoing-ip-address-for-w...

The NFS kernel module isn't installed, so you can't use EFS. (They suggest some 3rd party userland tool)

They expect you to set up VPN access with Wireguard for any connections to your containers. You can't just SCP your files to a volume. It's so much more hassle than connecting with kubectl cp or scp, especially if you're hoping to script things.

All that said, I'm happy to see competition in the "we'll run your docker image" space.

I've been using Fly for 2 years now. Overall I'm very happy and I recommend it to everyone.

If you just want to run a container in multiple regions with anycast, Fly is really the best option out there IMHO. Nothing comes close.

There are some rough edges for certain use cases but they keep polishing the service and the DX keeps getting better.

Personally the only features I'm missing today are:

- PG backups/snapshots. AFAIK these are coming in the form of virtual disk snapshots.

- Scale apps from zero to say 100 VMs like Cloud Run does. There's some autoscaling right now and the machines API, but still needs more polish. Specially for certain use cases like concurrent CPU heavy tasks (video encoding, etc). AFAIK some form of this is also coming in the next months.

Coming from a devops background, I kind of wish they didn't support arbitrary docker containers, as I found the experience fairly painful (going through one of their guides), and soured me from using Fly at all. Maybe it was the app's fault, but troubleshooting was not intuitive at all, especially on the networking side.

"Although everybody talks about how Kubernetes, for example, is complex and hard, most companies out there use it"

I am very skeptical of this claim.

I skimmed through their docs but couldn't find whether they:

* support for abstract apps, not only HTTP/web apps like heroku, let's say I want to deploy a SIP app

* support for HTTP/2 and potentially HTTP/3

If they do support these two I would say it's enough to be considered Heroku killer.

Has somebody used Render and Fly and can comment on the differences? I switched from Herokku to Render, and really like the good UX and docs. Never tried Fly though.

I tried Fly recently for the first time to run a few tiny servers for the challenges on protohackers.com

It has bee pretty great and painless. I have my own docker servers, and so on, but I don't have local registry setup, so dealing with getting images moved around, and all that hassle was going to be annoying.

Made a free fly.io account, did the `flyctl deploy` after making the config file and it just worked moments later. Really a nice flow.

Not sure yet if I would use it for anything else, but this was nice and easy so its definitely on my list of things to check for new projects.