Sign in

ERROR ACQUIRING THE STATE LOCK. NOBODY IS RUNNING TERRAFORM. OR ARE THEY?

Solvediacstateterraform25 replies · 528 views · started
Reply
  1. Diego Ferreira
    @diego.ferreira28

    im literally losing my mind here our depoly pipeline for the DNS module failed with this:

    Windows Update

    Windows Update
We couldn't complete the updates.
Undoing changes. Don't turn off your computer.

Error code: 0x800f0922

    A colleague hit Ctrl+C on a local apply last night cuz it was taikng too long 🙏 now nothing works. is the state broken? do i need to restore from backup??

    i don't even know whether the interrupted run changed any real resources 😂 What's the right order of operations here: unlock first, inspect first, restore a backup first? i'd love to avoid making a bad situation worse tonight.

    1
  2. Uma G.
    @umag683

    is versioning enabled on teh S3 with DynamoDB locking bucket or container? ??

    5 1 helpful
  3. Mateo Nolan
    @mateo.nolan-mq

    uhh is iac even worth bothering with or should i just leave it ??

    0 1 helpful
  5. Lea J.
    @lea.jansen

    lol same thing happened to me before unlocking, make suree the holder is really gone... the error shows Who, Operation and Created. check that CI job and that colleagues machine. if no process is running, its safe to run:

    terraform force-unlock <LOCK_ID>

    Force-unlock only removes teh lock entry, it does not change state contents.

    5 9 helpful
  6. Amara Holm
    @aholm

    have u checked your iac settings? mine were reset after an update lmk how it goes

    0 3 helpful
  7. Wren Vance
    @wrenv458

    yeahhh have u chceked your terraform settings? mine were reset after an update good luck man!

    0 5 helpful
  8. Ines
    @ibrandt

    everyone keeps saying this but nobody posts proof

    1 0 helpful
  10. Amara Holm
    @aholm

    oof after unlocking, run terraform plan first... if an interrupted apply had partly completed, teh plan will show resources that exist in teh cloud but not in state (or vice versa).. ;) fix that with terraform import or by re-running teh apply.

    4 1 helpful
  11. Ruben Rossi
    @ruben_shader

    honestly just reset the thermal paste and save yourself the pain hope that helps :)

    0 2 helpful
  12. Amara Holm
    @aholm

    yeahhh exact same thing happened to my mac last month worked for me anyway 🤷

    2 0 helpful
  13. Ruben Rossi
    @ruben_shader

    oof with S3 backends, enable bucket versioning. If the state file ever does get damaged you can restore the previous version from the bucket. Same idea applies to blob versioning in Azure Storage and object versioning in GCS.

    3 0 helpful
  14. Wren Vance
    @wrenv458

    hmmm never Ctrl+C an apply twice... the first interrupt lets Terraform finish in-flight operations and release teh lock; the second one kills it immediately and that's what leaves stale locks and half-written resourecs. worked for me anyway 🤷

    1 0 helpful
  15. Wes Wolfe
    @vectorotter

    If the lock persists on DynamoDB after force-unlock fails, you can delete the item for that state path from the lock table, but treat it as last resort and confirm nothing is running first.

    2 5 helpful
  17. Ines
    @ibrandt

    came here to say exactly this

    1 2 helpful
  18. Selin W.
    @cipher_sage

    hmmm +1 same here. spent half teh night on module and im still not done

    1 4 helpful
  19. Lea J.
    @lea.jansen

    @mateo.nolan-mq works on paper, not in real life

    4 2 helpful
  20. Una Reid
    @una_pixel

    wait what everyone keeps saying this but nobody posts proof idk maybe im wrong lol

    3 1 helpful

Reply to this topic

Sign in or create an account to reply.