Backend & APIs
Servers, databases, API design, queues and auth.
im literally losing my mind here were a startup backend team running a monolith plus two workers. Today they all talk JSON over HTTP and it works fine, but someone proposed moving internal traffic to gRPC: stronger contracts, generated clients, less hand…
i swear i installed it. pip install psycopg2 finished cleanly, pip list shows it, and when I run the app by hand eevrything works. then the systemd servcie starts it and I get this: Same server, same repo, same user (i think). Ive reinstalled three times and…
long time lurker first post lol i swear i installed it. pip install psycopg2 finished cleanly, pip list shows it, and when I run the app by hand everything works. Then teh Docker image starts it and I get this: same server, same repo, same user (I think).…
Let's swap war stories. Here's teh screen that haunted me for a weekend: app logs looked healthy, heatlh check passed locally, restarting fixed it for about twenty minutes and then teh 502s came back lol turned out to be something embarrassingly small.…
Im building an API where clients create orders with a POST 😂 Mobile clients on bad connections sometimes time out and retry, and weve ended up with duplicate orders a few times. customers notice duplicates immediaetly, and refunding them by hand is getting…
so i was just monitoring the logs and everything went completely sideways. the sync job was running fine all night, but suddenly the whole thing just collapsed and now the database is sitting there with half finished records... im honestly losing my mind…
our Ruby job runner has been rock solid for a year. since heavy concurrency it dies every few minutes with this: no exception, no stack trace, teh supervisor just restarts it. customers get random failures and i have literally nothing to grep for 🤦 how do…
say ur frontend sends fetch(url, { method: "POST", headers: { "Content Type": "application/json" }, credentials: "include" }) to an API on a different origin, and the browser blocks it. GETs are fine. The API does set…
ok so say your ORM backed endpoint is fine locally and slow in production. The query log shows the same SELECT ... WHERE id = ? repeated hundreds of times per request. locally everything feels instant cuz the test…
long time lurker first post lol I'm a backend dev, not a C person. Today my Ruby job runner started crashing after heavy concurrency, and all I get is: Our own code is plain high level code. I don't manage memory anywhere xD So how can it segfault? Is the…
Reproducible crash, at least. heavy concurrency kills the C++ API server instantly: i found a core file somewhere once but no idea what to do with it :D do i really need to learn gdb tonight? i can reproduce it in staging with the exact same input, which…
hmmmm ok this one is driving me crazy. after a reboot, a chunk of requests fail with this page: not all of them, maybe one in ten, the Spring Boot app health check is green the whole time, my manager asked if we turned it off and on again. we did... several…
small Flask ish API, nothing fancy. I moved it to a new box, created a venv, installed requirements, and hooked it up to the systemd service 🤷 instead of a running service I got this traceback: the weird part: if i source venv/bin/activate and run it, no…
not gonna lie this is driving me nuts our Python worker with a C extension has been rock solid for a year. since heavy concurrency it dies every few minutes with this: no exception, no stack trace, the supervisor just restarts it. customers get random…
Our Node API sits behind nginx on the same VM. It listens on 5000. It's been fine for months, but now after a reboot we get this: If I SSH in and curl localhost:5000 I get a 200 instantly. Restarting nginx fixes it for a while. I'm out of ideas and support…
Were about to design the internal APIs for a dozen intrenal services, greenfield, i keep going back and forth between plain REST with an OpenAPI spec and gRPC with protobuf. Performance is not our bottleneck right now, consistency is. What would u pick, and…
i swear i installed it. pip install dotenv finished cleanly, pip list shows it, and when i run the app by hand everything works. then the Docker image starts it and i get this: same server, same repo, same user (i think). ive reinstalled three times and even…
small Flask ish API, nothing fancy. I moved it to a new box, created a venv, installed requirements, and hooked it up to gunicorn. instead of a running service I got this traceback: The weird part: if I source venv/bin/activate and run it, no error at all,…
our Python worker with a C extension has been rock solid for a year. since heavy concurrency it dies every few minutes with this: No exception, no stack trace, teh supervisor just restarts it. Customers get random failures and i have literally nothing to…
ugh. honest question from a startup backend team. Our three servcies and a mobile app communicate over REST and every few weeks smth breaks because one side renamed a field. gRPC with generated code sounds like it would end that. But i also hear about…
im literally losing my mind here deploy night. Everything passed locally on Python 3.11, sooo i pushed and let the cron job pick it up. Two seconds later the logs filled with red: the package is right there in site packages, i can see the folder lmao my…
guys pls help reproducible crash, at least... the new OS image kills teh Ruby job runner instantly: i found a core file somewhere once but no idea what to do with it. Do i really need to learn gdb tonihgt? I can reproduce it in staging with the exact same…
ok so im a backend dev, not a C person, today my C++ API server started crashing after a library upgrade, and all I get is: Our own code is plain high level code. i dont manage memory anywhere. So how can it segfault? Is the server hardware dying? my manager…
long time lurker first post lol our Spring Boot app sits behind nginx on teh same VM. It listens on 5000. Its been fine for months, but now after a reboot we get this: if i SSH in and curl localhost:5000 i get a 200 instantly. restarting nginx fixes it for a…
so annoyed rn honest question from a four person team. Our three services and a mobile app communicate over REST and every few weeks something breaks because one side renamed a field :D gRPC with generated code sounds like it would end that. But i also hear…
