Automation & DevOps
Scripting, CI/CD, configuration management and observability.
im literally losing my mind here i might be wrong here, but Im losing my mind. I ran pip install requests, it said success. then I run my script: i ran pip install again and it says "Requirement already satisfied". so its installed AND not installed at the…
classic... pipelines are slow and most of the time goes to dependency installation and Docker builds, ive read about cacihng but Im nervous about flaky builds from stale caches. What do people typically cache in CI,…
our deploy pipeline for prod failed with this: A colleague hit Ctrl+C on a local apply last night cuz it was taking too long. now nothing works. is teh state broken? do i need to restore from backup? i dont even know whether the interrupted run changed any…
long time lurker first post lol just trying to start the web container with docker compose up on teh CI runner and i get: docker ps shows nothing running. i rebooted Docker. still the same. i need this running for a demo in an hour and i'm getting desperate.…
so annoyed rn i inherited a Azure DevOps agent nobody has touched in a year. today the test stage failed with ENOSPC and the disk looks like this: I'm scared to delete anything because I don't know what's safe ;) The whole team is blocked and looking at me.…
ok so every run of my Ansible playbook shows lots of "changed" tasks, so I cant tell what actually changed. The tasks use shell commands for things like adding lines to files and restarting services. what are the usual…
CI pipeline went red overnight. The job that spins up the dev API on a fresh Ubuntu box fails with: it passes on retry sometimes, which makes it even worse. i suspect leftovers from earlier jobs but i can't prove it. Our pipeline runs several jobs at once on…
hmmmm ok well. our small team has API keys and database passwords scattered around: some in CI variables, some in .env files on servers, and (Im embarrassed to say) a couple committed to a private repo at some point.…
guys pls help trying to apply a tiny chagne to staging and Terraform refuses: the lock info says it was created by a CI job that, as far as i can see, was cancelled hours ago. our backend is S3 with DynamoDB locking. i've read that force unlock can corrupt…
im literally losing my mind here our depoly pipeline for the DNS module failed with this: A colleague hit Ctrl+C on a local apply last night cuz it was taikng too long 🙏 now nothing works. is the state broken? do i need to restore from backup?? i don't even…
help!! just tyring to start teh Postgres container with docker compose up on the staging VM and I get: docker ps shows nothing running... i rebooted Docker... still the same... i need this running for a demo in an hour and Im getting desperate. im not a…
ok weird one we run Terraform from two separate pipelines that share the same S3 with DynamoDB locking state for prod. Since this week, one of them randomly fails with: Retrying usually works. Is this just a race, and how do ppl normally avoid it? our setup…
ugh. every pipeline on our GitLab runner staretd failing at image push. The error is basically no space left on device, and the monitoring shows this: we never stored anything big on that machine. I deleetd some old logs and freed maybe 2 GB, which was gone…
help!! i keep hearing that push based deploys are insecure cuz CI needs credentials to production. For a solo maintainer running a handful of VMs, is that a real concern? What did you gain or lose when switching to pull based? our auditors have started…
long time lurker first post lol CI pipeline went red overnight. the job that spins up the Redis container on a fresh Ubuntu box fails with: it passes on retry sometimes, which makes it even worse. I suspect leftovers from earlier jobs but I cant prove it.…
guys pls help every pipeline on our Jenkins agent satrted failing at the test stage. The error is basically no space left on device, and the monitoring shows this: we never stored anything big on that machine. i delteed some old logs and freed maybe 2 GB,…
not gonna lie this is driving me nuts this worked fine last week... today, on a fresh Ubuntu box, teh web container dies right away: i didnt change the compose file. port 27017. ive checked Task Manager and htop and i cant find anything obvious :) is Docker…
help!! starting fresh with on prem servers. I can either do classic CI/CD where the pipeline deploys, or set up a GitOps controller watching a config repo. a small agency, limited time. What would u choose today? i have about two weeks to set this up before…
not gonna lie this is driving me nuts this worked fine last week. Today, on a fresh Ubuntu box, the web container dies right away: i didnt change the compose file. Port 27017 lmao Ive checked Task Manager and htop and i cant find anything obvious. Is Docker…
Weird one. The GitLab runner disk is full: but when i run du sh / teh numbers don't add up to anywhere near the disk size. It's like teh space is invisible. Builds keep failing at dokcer build. i checked teh obvious folders, home directories and /tmp, and…
this worked fine last week, today, on my laptop, the reverse proxy dies right away: i didn't change the compose file. Port 6379. i've checked Task Manager and htop and i can't find anything obvious. Is Docker keeping a ghost reservation? ive already run…
weird one. the Jenkins agent disk is full: But when i run du sh / the numbers dont add up to anywhere near the disk size... its like teh space is invisible... builds keep failing at artifact upload. i checked the obvious folders, home directories and /tmp,…
This worked fine last week. Today, on a fresh Ubuntu box, the reverse proxy dies right away: i didn't change the compose file. Port 3000. i've checked Task Manager and htop and i can't find anythnig obvious. Is Docker keeping a ghost reservation?? ive…
We run Terraform from two separate pipelines that share teh same GCS bucket state for the shared dev environment. Since this week, one of them randomly fails with: Retrying usually works. Is this just a race, and how do people normally avoid it? Our setup…
hmmmm ok i inherited a Jenkins agent nobody has touched in a year. Today artifact upload failed with ENOSPC and teh disk looks like this: Im scared to delete anything because i dont know whats safe. The whole team is blocked and looking at me. theres no…
