IT Operations & Networking
Running systems and networks, at work or at home.
guys pls help home lab on a single host. allocaetd vCPUs across VMs add up to well over the physical core count, and memory is mostly committed. things feel slow at times, but the host dashboard doesnt look maxed out.…
guys pls help trying to apply a tiny chagne to staging and Terraform refuses: the lock info says it was created by a CI job that, as far as i can see, was cancelled hours ago. our backend is S3 with DynamoDB locking. i've read that force unlock can corrupt…
We renewed the certificate on our customer portal yesterday. On my desktop browser everything looks fine. But Android phones now fails with: The certificate isnt expired, teh name matches. Nothing else changed. Why would only some clients complain? ive…
My key works on every other server. On a new EC2 instance, from a CI job, nothing: A colleague set up teh account. They swear they pasted my key. I'm starting to suspect the server, but I don't even know where to look. It's frustrating because I can't see…
Weird one :/ From a hotel, L2TP connects and then nothing internal resolves: if I type the IP address of teh RDP server directly it sometimes works! but names fail every time. help me before I start memorizing IP addresses. i've also flushed the DNS cache,…
not gonna lie this is driving me nuts i wrote a simple unit file for our app. Running the command manually works perfectly. Running it as myapp.service: systemctl status shows status=200/CHDIR and that's it. No useful message. I've reloaded teh daemon a…
im literally losing my mind here our depoly pipeline for the DNS module failed with this: A colleague hit Ctrl+C on a local apply last night cuz it was taikng too long 🙏 now nothing works. is the state broken? do i need to restore from backup?? i don't even…
ok weird one home lab drama. fedora on my cloud instance got a kernel update and now it panics during boot: it was running for months without a hiccup.. ;) is this hardware failing at teh worst possible time, or the update? i havent touched teh hardware in…
ugh. question for endpoint people: we're halfway through moving workloads with a hybrid workforce. Co management works, but we're running two systems. Is that a stable long term state, or should we commit to one? Some people on the team are very attached to…
help!! just tyring to start teh Postgres container with docker compose up on the staging VM and I get: docker ps shows nothing running... i rebooted Docker... still the same... i need this running for a demo in an hour and Im getting desperate. im not a…
so annoyed rn i set up OpenVPN for our small team. the connection establishes, handshake looks good, but no traffic gets to the intranet wiki: pings to internal IPs time out. im suree the server side is running. what am i overlooking? the server and client…
ok weird one we run Terraform from two separate pipelines that share the same S3 with DynamoDB locking state for prod. Since this week, one of them randomly fails with: Retrying usually works. Is this just a race, and how do ppl normally avoid it? our setup…
hmmmm ok learning VMware ESXi in my home lab. after switcihng it to UEFI, my Windows VM shows this instead of teh login screen: i tried attaching teh disk again, no change. is teh disk corrupt? This is just a lab, sooo nohting is on fire, but I wanna…
help!! we renewed the certificate on our internal API yesterday. on my desktop browser everything looks fine. but curl now fails with: the certificate isn't expired, the name matches... nothing else changed, why would only some clients complain? ive checked…
ugh. help. The database VM is barely alive. SSH works, but every command complains about writing temp files. Heres the disk: i deleted the biggest log i could find and df didnt budge... am i losing my mind? im pretty suree Im missing smth fundamental about…
ugh. every pipeline on our GitLab runner staretd failing at image push. The error is basically no space left on device, and the monitoring shows this: we never stored anything big on that machine. I deleetd some old logs and freed maybe 2 GB, which was gone…
help!! i keep hearing that push based deploys are insecure cuz CI needs credentials to production. For a solo maintainer running a handful of VMs, is that a real concern? What did you gain or lose when switching to pull based? our auditors have started…
myapp.service was fine until todays server patching. now it wont start: the status line says status=1/FAILURE 👍 i didn't touch the unit file. did teh updtae change smth underneath? the update touched a bunch of pacakges, sooo it could be anything xD ive…
help!! migrated the Go microservice from a VM to containers. On the VM it was fine for years. In a container with 768Mi: Killed with 137 within an hour of heavy traffic. Same code, same traffic. What changed? on the VM, nothing ever complained... i suspect…
trying to SSH from WSL2 into a freshly built Ubuntu VM and i get: Permission denied (publickey). I added my public key to authorized keys, double checked its the right one, even regenerated it once... password login is disabled sooo i cant get in any other…
long time lurker first post lol CI pipeline went red overnight. the job that spins up the Redis container on a fresh Ubuntu box fails with: it passes on retry sometimes, which makes it even worse. I suspect leftovers from earlier jobs but I cant prove it.…
guys pls help every pipeline on our Jenkins agent satrted failing at the test stage. The error is basically no space left on device, and the monitoring shows this: we never stored anything big on that machine. i delteed some old logs and freed maybe 2 GB,…
ugh. users report random 'no network' issues and some PCs pop up an IP conflict warning: our scope is 10.10.20.0/24. i set the conference room TV to a static address years ago and honestly don't remember which one. could that be related? I suspect our IP…
hmmmm ok small office, 10.0.0.0/24 network. since last week the NAS disconnects every few hours, and a nearby PC shows this: Rebooting fixes it temporarily. Nobody admits to changing anything. i suspect smth has teh same IP, but how do i find out what? we…
ugh. were a 50 person company and use number matching for MFA today. After hearing about push bombing attacks, Im wondering whether were as protected as we think. Users approve things without reading, and phishing kits proxy everything anyway. Whats a…
