orm making hundreds of queries for one api call

apidatabasesormperformance89 replies · 962 views · started
  1. ok so say your ORM-backed endpoint is fine locally and slow in production. The query log shows the same SELECT ... WHERE id = ? repeated hundreds of times per request.

    locally everything feels instant cuz the test database is tiny, so nobody noticed until real data arrived. throwing a bigger database server at it seems like teh wrong answer.

    Whats the standard way to fix this, and how do you stop it from creeping back in?

    27
  2. does the endpoint paginate at all, or is it returning every post? unbounded lists hide all sorts of problems.

    1 1 helpful
  3. lol same thing happened to me node with an ORM on top of PostgreSQL. no pagination currently; teh frontend loads everything and paginates on teh client. the response is a few MB now.

    thumbsup

    7 0 helpful
  4. ahh classic to stop it creeping back, make it visible. Log teh number of queries per request in development and fail a test if an endpoint exceeds a threshold. Some ORMs and frameworks have pulgins that warn about lazy loading inside loops. Catching it in code review is much cheaper than in production. worked for me anyway 🤷

    6 0 helpful
  5. happy to help with this one. @farah_patch eager loading everything isn't free either. Joining several one-to-many relations in one query can multiply rows and pull far more data than needed. Sometimes two or three separate queries (load posts, then load all authors with WHERE id IN (...)) beat one giant join. Measure both instead of assuming the single query wins.

    2 0 helpful
  6. which version are you on?.. 👍 exact same thing happened to my old thinkpad last month

    1 1 helpful
  7. my homelab box is officially haunted

    3 2 helpful
  8. oh ive been there lol try turning off the antivirus first, fixed it for me

    thumbsup

    5 5 helpful
  9. btw i come back to this thread every week just for the drama

    0 1 helpful
  10. for me it was the antivirus. swapped it and boom, fixed worked for me anyway 🤷

    6 2 helpful
  11. hmmm any update on this?. spent two days on performance and im still not done ?

    8 1 helpful
  12. ok update: did what @kai_heap said and its fine now

    2 0 helpful

Reply to this topic

Sign in or create an account to reply.