Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Stack is slow to print "> using precompiled package" #6680

Open
chreekat opened this issue Jan 29, 2025 · 4 comments
Open

Stack is slow to print "> using precompiled package" #6680

chreekat opened this issue Jan 29, 2025 · 4 comments

Comments

@chreekat
Copy link
Member

The first ~dozen are fast, but eventually it slows down.

I ran stack -v build. Below are all the debug messages between three of the "> using precompiled package" lines. (The entire output is also attached). If you look at the timestamps, indexed-traversable is obviously fast, and ansi-terminal is very slow by comparison.

All of the other slow packages also have a bunch of debug output and sqlite activity.

This behavior showed up unexpectedly when I changed one of my dependencies. I added an extra-dep that overrode version of amazonka that was being used from the snapshot. Although this won't help with repro, I'll show the change that triggered the behavior for context:

extra-deps:
 # <snip>

 # This amazonka includes the unreleased patch to support Cloudflare, which kinda
 # has a bug. See https://github.com/brendanhay/amazonka/issues/975 for details. 
 - github: brendanhay/amazonka                                                   
   commit: '85e0289f8dc23c54b00f7f1a09845be7e032a1eb'                            
   subdirs:                                                                      
     # This has the patch                                                        
     - lib/amazonka-core                                                         
     # This one has a build failure if taken from lts-22.7                       
     - lib/services/amazonka-s3                                                  
+    # Same                                                                      
+    - lib/amazonka

Before that change, the project built quickly and didn't print any "using precompiled package" messages, slowly or otherwise.

Here's the log "snippet":

2025-01-29 13:13:36.855748: [info] hourglass                        > using precompiled package
2025-01-29 13:13:36.864073: [debug] (SQL) SELECT "id","platform_ghc_dir","compiler","cabal_version","package_key","options_hash","haddock","library" FROM "precompiled_cache" WHERE "platform_ghc_dir"=? AND "compiler"=? AND "cabal_version"=? AND "package_key"=? AND "options_hash"=? AND "haddock"=?; [PersistText "x86_64-linux-nix/",PersistText "ghc-9.6.4",PersistText "3.10.1.0",PersistText "181cff1f160115b7ed52b9da8f4f9d10e7042821f74b23381044661120f180e6,625",PersistByteString "8^g{\239X\158\166\208\240\149\195Q\143,\164<:\DELO\DC2\179<\NUL\137\243XL\189\237\&6\210",PersistBool False]
2025-01-29 13:13:36.865894: [debug] (SQL) SELECT "id", "precompiled_cache_id", "sub_lib" FROM "precompiled_cache_sub_lib" WHERE ("precompiled_cache_id"=?); [PersistInt64 7152]
2025-01-29 13:13:36.866077: [debug] (SQL) SELECT "id", "precompiled_cache_id", "exe" FROM "precompiled_cache_exe" WHERE ("precompiled_cache_id"=?); [PersistInt64 7152]
2025-01-29 13:13:36.866192: [info] indexed-traversable              > using precompiled package
2025-01-29 13:13:36.870933: [debug] Reading package config:
/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/lib/ghc-9.6.4/lib/package.conf.d/Cabal-syntax-3.10.1.0.conf
2025-01-29 13:13:36.874497: [debug] Reading package config:
/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/lib/ghc-9.6.4/lib/package.conf.d/array-0.5.6.0.conf
2025-01-29 13:13:36.875350: [debug] Reading package config:
/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/lib/ghc-9.6.4/lib/package.conf.d/base-4.18.2.0.conf
2025-01-29 13:13:36.879509: [debug] Reading package config:
/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/lib/ghc-9.6.4/lib/package.conf.d/binary-0.8.9.1.conf
2025-01-29 13:13:36.880287: [debug] Reading package config:
/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/lib/ghc-9.6.4/lib/package.conf.d/bytestring-0.11.5.3.conf
2025-01-29 13:13:36.881788: [debug] Reading package config:
/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/lib/ghc-9.6.4/lib/package.conf.d/containers-0.6.7.conf
2025-01-29 13:13:36.883055: [debug] Reading package config:
/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/lib/ghc-9.6.4/lib/package.conf.d/deepseq-1.4.8.1.conf
2025-01-29 13:13:36.883785: [debug] Reading package config:
/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/lib/ghc-9.6.4/lib/package.conf.d/directory-1.3.8.1.conf
2025-01-29 13:13:36.884549: [debug] Reading package config:
/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/lib/ghc-9.6.4/lib/package.conf.d/exceptions-0.10.7.conf
2025-01-29 13:13:36.885430: [debug] Reading package config:
/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/lib/ghc-9.6.4/lib/package.conf.d/filepath-1.4.200.1.conf
2025-01-29 13:13:36.886677: [debug] Reading package config:
/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/lib/ghc-9.6.4/lib/package.conf.d/ghc-9.6.4.conf
2025-01-29 13:13:36.896734: [debug] Reading package config:
/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/lib/ghc-9.6.4/lib/package.conf.d/ghc-bignum-1.3.conf
2025-01-29 13:13:36.897889: [debug] Reading package config:
/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/lib/ghc-9.6.4/lib/package.conf.d/ghc-boot-9.6.4.conf
2025-01-29 13:13:36.898938: [debug] Reading package config:
/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/lib/ghc-9.6.4/lib/package.conf.d/ghc-boot-th-9.6.4.conf
2025-01-29 13:13:36.899914: [debug] Reading package config:
/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/lib/ghc-9.6.4/lib/package.conf.d/ghc-compact-0.1.0.0.conf
2025-01-29 13:13:36.900630: [debug] Reading package config:
/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/lib/ghc-9.6.4/lib/package.conf.d/ghc-heap-9.6.4.conf
2025-01-29 13:13:36.901454: [debug] Reading package config:
/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/lib/ghc-9.6.4/lib/package.conf.d/ghc-prim-0.10.0.conf
2025-01-29 13:13:36.902384: [debug] Reading package config:
/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/lib/ghc-9.6.4/lib/package.conf.d/ghci-9.6.4.conf
2025-01-29 13:13:36.903193: [debug] Reading package config:
/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/lib/ghc-9.6.4/lib/package.conf.d/haskeline-0.8.2.1.conf
2025-01-29 13:13:36.904498: [debug] Reading package config:
/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/lib/ghc-9.6.4/lib/package.conf.d/hpc-0.6.2.0.conf
2025-01-29 13:13:36.905128: [debug] Reading package config:
/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/lib/ghc-9.6.4/lib/package.conf.d/integer-gmp-1.1.conf
2025-01-29 13:13:36.905895: [debug] Reading package config:
/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/lib/ghc-9.6.4/lib/package.conf.d/libiserv-9.6.4.conf
2025-01-29 13:13:36.906461: [debug] Reading package config:
/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/lib/ghc-9.6.4/lib/package.conf.d/mtl-2.3.1.conf
2025-01-29 13:13:36.907264: [debug] Reading package config:
/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/lib/ghc-9.6.4/lib/package.conf.d/parsec-3.1.16.1.conf
2025-01-29 13:13:36.908396: [debug] Reading package config:
/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/lib/ghc-9.6.4/lib/package.conf.d/pretty-1.1.3.6.conf
2025-01-29 13:13:36.909078: [debug] Reading package config:
/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/lib/ghc-9.6.4/lib/package.conf.d/process-1.6.17.0.conf
2025-01-29 13:13:36.910001: [debug] Reading package config:
/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/lib/ghc-9.6.4/lib/package.conf.d/rts-1.0.2.conf
2025-01-29 13:13:36.911928: [debug] Reading package config:
/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/lib/ghc-9.6.4/lib/package.conf.d/stm-2.5.1.0.conf
2025-01-29 13:13:36.912827: [debug] Reading package config:
/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/lib/ghc-9.6.4/lib/package.conf.d/system-cxx-std-lib-1.0.conf
2025-01-29 13:13:36.913230: [debug] Reading package config:
/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/lib/ghc-9.6.4/lib/package.conf.d/template-haskell-2.20.0.0.conf
2025-01-29 13:13:36.914358: [debug] Reading package config:
/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/lib/ghc-9.6.4/lib/package.conf.d/terminfo-0.4.1.6.conf
2025-01-29 13:13:36.915189: [debug] Reading package config:
/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/lib/ghc-9.6.4/lib/package.conf.d/text-2.0.2.conf
2025-01-29 13:13:36.917001: [debug] Reading package config:
/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/lib/ghc-9.6.4/lib/package.conf.d/time-1.12.2.conf
2025-01-29 13:13:36.918542: [debug] Reading package config:
/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/lib/ghc-9.6.4/lib/package.conf.d/transformers-0.6.1.0.conf
2025-01-29 13:13:36.919677: [debug] Reading package config:
/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/lib/ghc-9.6.4/lib/package.conf.d/unix-2.8.4.0.conf
2025-01-29 13:13:36.921221: [debug] Reading package config:
/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/lib/ghc-9.6.4/lib/package.conf.d/xhtml-3000.2.2.1.conf
2025-01-29 13:13:36.922314: [debug] Run process: /nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/bin/ghc-pkg-9.6.4 --no-user-package-db --package-db=/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb/ register --force /home/b/.stack/snapshots/x86_64-linux-nix/39a92b495c4e9c5db79ef04867654e8cd6a2045e38545a7278632e1a009b9043/9.6.4/pkgdb/ansi-terminal-types-0.11.5-7QpUULXXpCL6RLITE1JUOk.conf
2025-01-29 13:13:37.032091: [debug] Process finished in 110ms: /nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/bin/ghc-pkg-9.6.4 --no-user-package-db --package-db=/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb/ register --force /home/b/.stack/snapshots/x86_64-linux-nix/39a92b495c4e9c5db79ef04867654e8cd6a2045e38545a7278632e1a009b9043/9.6.4/pkgdb/ansi-terminal-types-0.11.5-7QpUULXXpCL6RLITE1JUOk.conf
2025-01-29 13:13:37.032372: [debug] Run process: /nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/bin/ghc-pkg-9.6.4 --user --no-user-package-db --package-db /home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb describe --simple-output ansi-terminal-types --expand-pkgroot
2025-01-29 13:13:37.033551: [debug] Unregistering from
/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb/:
blaze-html-0.9.1.2.
2025-01-29 13:13:37.034926: [debug] Reading package config:
/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb/OneTuple-0.4.1.1-413yz2qDv5HLK54zgzfSJ3.conf
2025-01-29 13:13:37.036830: [debug] Reading package config:
/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb/Only-0.1-2OJr6Dq0xOm75uZ80XlI7x.conf
2025-01-29 13:13:37.038953: [debug] Reading package config:
/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb/StateVar-1.2.2-LMwI0SGrUtP1MlPXMPkYAS.conf
2025-01-29 13:13:37.072635: [debug] Reading package config:
/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb/annotated-wl-pprint-0.7.0-2QW85nLxF0QDC21sjKmQFb.conf
2025-01-29 13:13:37.074845: [debug] Reading package config:
/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb/appar-0.1.8-2UYkbPOz3CVDU6UBsF1opz.conf
2025-01-29 13:13:37.076899: [debug] Reading package config:
/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb/auto-update-0.1.6-ADHZym4WmY5G8RwewI8v4E.conf
2025-01-29 13:13:37.078920: [debug] Reading package config:
/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb/base-compat-0.13.1-G7NGIqA1Gm7N8bViVREwP.conf
2025-01-29 13:13:37.082818: [debug] Reading package config:
/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb/base-orphans-0.9.1-KAZ8G7uKs3r7jrb0e6niG4.conf
2025-01-29 13:13:37.084927: [debug] Reading package config:
/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb/base16-bytestring-1.0.2.0-30XYP7z649IagQeQTppof.conf
2025-01-29 13:13:37.087094: [debug] Reading package config:
/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb/base64-bytestring-1.2.1.0-K1erZvkjHObEU0kBt3tz3o.conf
2025-01-29 13:13:37.089290: [debug] Reading package config:
/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb/basement-0.0.16-DD6H8SGD4AW2EysJRRfDZR.conf
2025-01-29 13:13:37.094395: [debug] Reading package config:
/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb/blaze-builder-0.4.2.3-KqOYhpYf1HiJNpffbEN3Kt.conf
2025-01-29 13:13:37.098232: [debug] Reading package config:
/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb/bsb-http-chunked-0.0.0.4-IC9ZPBN6tEZ1OZx3ALxxCt.conf
2025-01-29 13:13:37.101419: [debug] Reading package config:
/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb/byteable-0.1.1-4CNQdo3Prm7AURHENelJp7.conf
2025-01-29 13:13:37.104815: [debug] Reading package config:
/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb/byteorder-1.0.4-KgHnyAnKoyVGvIK2rOLowO.conf
2025-01-29 13:13:37.107845: [debug] Reading package config:
/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb/cabal-doctest-1.0.9-BNPaF3kbrctJvYhpvWMgHi.conf
2025-01-29 13:13:37.110974: [debug] Reading package config:
/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb/call-stack-0.4.0-5ydnsiRoTiXIBqpTNw0DxP.conf
2025-01-29 13:13:37.114458: [debug] Reading package config:
/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb/cereal-0.5.8.3-9sqHuqlzPicJDqNz9TSzXN.conf
2025-01-29 13:13:37.114772: [debug] Process finished in 82ms: /nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/bin/ghc-pkg-9.6.4 --user --no-user-package-db --package-db /home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb describe --simple-output ansi-terminal-types --expand-pkgroot
2025-01-29 13:13:37.116909: [debug] (SQL) SELECT "id","directory","type","pkg_src","active","path_env_var","haddock" FROM "config_cache" WHERE "directory"=? AND "type"=?; [PersistText "/home/b/worktrees/stackage-server/b/cabal-nix/.stack-work/install/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/",PersistText "lib:ansi-terminal-types-0.11.5-7QpUULXXpCL6RLITE1JUOk"]
2025-01-29 13:13:37.118743: [debug] Reading package config:
/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb/clock-0.8.4-3bdbhUDgwcB9MPcgrlycwm.conf
2025-01-29 13:13:37.120507: [debug] (SQL) INSERT INTO "config_cache"("directory","type","pkg_src","active","path_env_var","haddock") VALUES(?,?,?,?,?,?); [PersistText "/home/b/worktrees/stackage-server/b/cabal-nix/.stack-work/install/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/",PersistText "lib:ansi-terminal-types-0.11.5-7QpUULXXpCL6RLITE1JUOk",PersistText "upstream",PersistBool True,PersistText "/nix/store/x9d49vaqlrkw97p9ichdwrnbh013kq7z-bash-interactive-5.2p37/bin:/nix/store/wnl9qpnhayry14lhcbdafhadsjwsdr6p-patchelf-0.15.0/bin:/nix/store/k09kpzc49l0q7cikc7jbm5vhangikl6a-postgresql-16.6-dev/bin:/nix/store/s5gx0m0pxn0lgwp381jmpsbb7hcmyqqj-postgresql-16.6/bin:/nix/store/2gc1mdxn6whvjscqibxkfaamfhspbmcd-pkg-config-wrapper-0.29.2/bin:/nix/store/r3br2rakxvsds5nsk362iix79hfxzq7b-haskell-language-server-2.9.0.0/bin:/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/bin:/nix/store/7793j1am5j1pj6d3p7fr8z9q671m6wkc-git-2.47.2/bin:/nix/store/888bkaqdpfpx72dd8bdc69qsqlgbhcvf-gcc-wrapper-13.3.0/bin:/nix/store/62qjb50708fdhb4f2y7zxyqr1afir4fk-gcc-13.3.0/bin:/nix/store/29mb4q8b5306f4gk2wh38h0c1akb0n97-glibc-2.40-36-bin/bin:/nix/store/6wgd8c9vq93mqxzc7jhkl86mv6qbc360-coreutils-9.5/bin:/nix/store/qlzvmgr8w9prdlyys7irqf86p7bndf5b-binutils-wrapper-2.43.1/bin:/nix/store/vk4mlknqk9yjbqa68a7rvpfxfdw3rad7-binutils-2.43.1/bin:/nix/store/k3a7dzrqphj9ksbb43i24vy6inz8ys51-ncurses-6.4.20221231/bin:/nix/store/6wgd8c9vq93mqxzc7jhkl86mv6qbc360-coreutils-9.5/bin:/nix/store/r99d2m4swgmrv9jvm4l9di40hvanq1aq-findutils-4.10.0/bin:/nix/store/3sln66ij8pg114apkd8p6nr04y37q5z2-diffutils-3.10/bin:/nix/store/yq39xdwm4z0fhx7dsm8mlpgvcz3vbfg3-gnused-4.9/bin:/nix/store/vniy1y5n8g28c55y7788npwc4h09fh7c-gnugrep-3.11/bin:/nix/store/scgfwh3z1s3l2vhvyjsgfgx5ql552sls-gawk-5.3.1/bin:/nix/store/0wqn2k3v5xzrc9rwinijdyr2ywwl82x4-gnutar-1.35/bin:/nix/store/5y240z436gf3rwmkwbhn1a17pqw509w4-gzip-1.13/bin:/nix/store/1yyryxp7mh7zsciapi8f1n0mnxkigmf8-bzip2-1.0.8-bin/bin:/nix/store/hbzw8k8ygv6bfzvsvnd4gb8qmm8xjbvn-gnumake-4.4.1/bin:/nix/store/gwgqdl0242ymlikq9s9s62gkp5cvyal3-bash-5.2p37/bin:/nix/store/rfrjws98w6scfx7m63grb0m6sg925ahd-patch-2.7.6/bin:/nix/store/4i4mjaf7z6gddspar487grxk5k1j4dcd-xz-5.6.3-bin/bin:/nix/store/9wbpsj6ksd16x1qdqs29xli1dpz3fnl0-file-5.45/bin",PersistBool False]
2025-01-29 13:13:37.122522: [debug] Reading package config:
/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb/cmark-gfm-0.2.6-E9kBFwt6gFYKe6gOT9gPEr.conf
2025-01-29 13:13:37.126078: [debug] Reading package config:
/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb/cmdargs-0.10.22-83WnG7g72QcDcTY9CeATsx.conf
2025-01-29 13:13:37.126328: [debug] (SQL) SELECT "id" FROM "config_cache" WHERE _ROWID_=last_insert_rowid(); []
2025-01-29 13:13:37.127560: [debug] (SQL) DELETE FROM "config_cache_dir_option" WHERE ("config_cache_id"=?) AND (1=2); [PersistInt64 594]
2025-01-29 13:13:37.130847: [debug] (SQL) INSERT INTO "config_cache_dir_option"("config_cache_id","index","option") VALUES (?,?,?),(?,?,?),(?,?,?),(?,?,?),(?,?,?),(?,?,?),(?,?,?),(?,?,?),(?,?,?),(?,?,?),(?,?,?),(?,?,?); [PersistInt64 594,PersistInt64 0,PersistText "--user",PersistInt64 594,PersistInt64 1,PersistText "--package-db=clear",PersistInt64 594,PersistInt64 2,PersistText "--package-db=global",PersistInt64 594,PersistInt64 3,PersistText "--package-db=/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb",PersistInt64 594,PersistInt64 4,PersistText "--libdir=/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/lib",PersistInt64 594,PersistInt64 5,PersistText "--bindir=/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/bin",PersistInt64 594,PersistInt64 6,PersistText "--datadir=/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/share",PersistInt64 594,PersistInt64 7,PersistText "--libexecdir=/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/libexec",PersistInt64 594,PersistInt64 8,PersistText "--sysconfdir=/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/etc",PersistInt64 594,PersistInt64 9,PersistText "--docdir=/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/doc/ansi-terminal-types-0.11.5",PersistInt64 594,PersistInt64 10,PersistText "--htmldir=/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/doc/ansi-terminal-types-0.11.5",PersistInt64 594,PersistInt64 11,PersistText "--haddockdir=/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/doc/ansi-terminal-types-0.11.5"]
2025-01-29 13:13:37.131767: [debug] Reading package config:
/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb/colour-2.3.6-L6nsTBN10g6LEOaPT1DhwH.conf
2025-01-29 13:13:37.135293: [debug] Reading package config:
/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb/base-compat-batteries-0.13.1-HJCVv1SaM2B87GijOeLP77.conf
2025-01-29 13:13:37.135519: [debug] (SQL) DELETE FROM "config_cache_no_dir_option" WHERE ("config_cache_id"=?) AND (1=2); [PersistInt64 594]
2025-01-29 13:13:37.137807: [debug] (SQL) INSERT INTO "config_cache_no_dir_option"("config_cache_id","index","option") VALUES (?,?,?),(?,?,?),(?,?,?),(?,?,?),(?,?,?),(?,?,?),(?,?,?),(?,?,?),(?,?,?),(?,?,?),(?,?,?),(?,?,?),(?,?,?),(?,?,?),(?,?,?),(?,?,?),(?,?,?),(?,?,?),(?,?,?),(?,?,?),(?,?,?),(?,?,?); [PersistInt64 594,PersistInt64 0,PersistText "--dependency=base=base-4.18.2.0",PersistInt64 594,PersistInt64 1,PersistText "--dependency=colour=colour-2.3.6-L6nsTBN10g6LEOaPT1DhwH",PersistInt64 594,PersistInt64 2,PersistText "--extra-include-dirs=/nix/store/06q0p7bhn2ffxxya20rrfqdib3h32csn-zlib-1.3.1-dev/include",PersistInt64 594,PersistInt64 3,PersistText "--extra-include-dirs=/nix/store/k09kpzc49l0q7cikc7jbm5vhangikl6a-postgresql-16.6-dev/include",PersistInt64 594,PersistInt64 4,PersistText "--extra-include-dirs=/nix/store/2gc1mdxn6whvjscqibxkfaamfhspbmcd-pkg-config-wrapper-0.29.2/include",PersistInt64 594,PersistInt64 5,PersistText "--extra-include-dirs=/nix/store/r3br2rakxvsds5nsk362iix79hfxzq7b-haskell-language-server-2.9.0.0/include",PersistInt64 594,PersistInt64 6,PersistText "--extra-include-dirs=/nix/store/3skhm0g57g1g3cynnxmics388jvg163w-nss-cacert-3.107/include",PersistInt64 594,PersistInt64 7,PersistText "--extra-include-dirs=/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/include",PersistInt64 594,PersistInt64 8,PersistText "--extra-include-dirs=/nix/store/7793j1am5j1pj6d3p7fr8z9q671m6wkc-git-2.47.2/include",PersistInt64 594,PersistInt64 9,PersistText "--extra-include-dirs=/nix/store/888bkaqdpfpx72dd8bdc69qsqlgbhcvf-gcc-wrapper-13.3.0/include",PersistInt64 594,PersistInt64 10,PersistText "--extra-include-dirs=/nix/store/awr474gcqxa74764fkx8yvljlp09xp12-gmp-with-cxx-6.3.0-dev/include",PersistInt64 594,PersistInt64 11,PersistText "--extra-lib-dirs=/nix/store/l5g2v1jgfyf3j0jp9iv5b79fi8yrwzpp-zlib-1.3.1/lib",PersistInt64 594,PersistInt64 12,PersistText "--extra-lib-dirs=/nix/store/il7k91hccyf2ca5dczq48nam8nn13zqk-postgresql-16.6-lib/lib",PersistInt64 594,PersistInt64 13,PersistText "--extra-lib-dirs=/nix/store/2gc1mdxn6whvjscqibxkfaamfhspbmcd-pkg-config-wrapper-0.29.2/lib",PersistInt64 594,PersistInt64 14,PersistText "--extra-lib-dirs=/nix/store/r3br2rakxvsds5nsk362iix79hfxzq7b-haskell-language-server-2.9.0.0/lib",PersistInt64 594,PersistInt64 15,PersistText "--extra-lib-dirs=/nix/store/3skhm0g57g1g3cynnxmics388jvg163w-nss-cacert-3.107/lib",PersistInt64 594,PersistInt64 16,PersistText "--extra-lib-dirs=/nix/store/rmlaizhjgwfq49v6j12z06vwck9nl3hw-ghc-9.6.4/lib",PersistInt64 594,PersistInt64 17,PersistText "--extra-lib-dirs=/nix/store/7793j1am5j1pj6d3p7fr8z9q671m6wkc-git-2.47.2/lib",PersistInt64 594,PersistInt64 18,PersistText "--extra-lib-dirs=/nix/store/888bkaqdpfpx72dd8bdc69qsqlgbhcvf-gcc-wrapper-13.3.0/lib",PersistInt64 594,PersistInt64 19,PersistText "--extra-lib-dirs=/nix/store/cpyk8ksm7c79l0nabd576s80wngdzix5-gmp-with-cxx-6.3.0/lib",PersistInt64 594,PersistInt64 20,PersistText "--exact-configuration",PersistInt64 594,PersistInt64 21,PersistText "--ghc-option=-fhide-source-paths"]
2025-01-29 13:13:37.142916: [debug] Reading package config:
/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb/conduit-combinators-1.3.0-IO4AjbfkVSt2tnnQKgF77V.conf
2025-01-29 13:13:37.146657: [debug] Reading package config:
/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb/contravariant-1.5.5-KzhCBNYXMkYFS6JTj2kIRH.conf
2025-01-29 13:13:37.144109: [debug] (SQL) DELETE FROM "config_cache_dep" WHERE ("config_cache_id"=?) AND (1=2); [PersistInt64 594]
2025-01-29 13:13:37.147917: [debug] (SQL) INSERT INTO "config_cache_dep"("config_cache_id","ghc_pkg_id") VALUES (?,?),(?,?); [PersistInt64 594,PersistText "base-4.18.2.0",PersistInt64 594,PersistText "colour-2.3.6-L6nsTBN10g6LEOaPT1DhwH"]
2025-01-29 13:13:37.150270: [debug] Reading package config:
/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb/cryptohash-md5-0.11.101.0-LlbmVuBbccu9tXNi6TxuHk.conf
2025-01-29 13:13:37.153619: [debug] Reading package config:
/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb/cryptohash-sha1-0.11.101.0-52VKih8lnvCJbZmLfShIh1.conf
2025-01-29 13:13:37.157690: [debug] Reading package config:
/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb/blaze-markup-0.8.3.0-2vtdi1m30QR7YYEGi2s89A.conf
2025-01-29 13:13:37.160985: [debug] Reading package config:
/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb/cryptohash-sha256-0.11.102.1-EdprTK4ljZA2CfmacPS9ny.conf
2025-01-29 13:13:37.164851: [debug] Reading package config:
/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb/data-default-class-0.1.2.0-DWa0eFdALruJzXj3E1ovxz.conf
2025-01-29 13:13:37.168115: [debug] Reading package config:
/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb/digest-0.0.2.0-8Wrp4BM4ICxM46nCzZ1IN.conf
2025-01-29 13:13:37.171295: [debug] Reading package config:
/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb/dlist-1.0-8ZSsfhwGOfJIXQNNI9Fv2.conf
2025-01-29 13:13:37.174685: [debug] Reading package config:
/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb/HUnit-1.6.2.0-2aRzyZJbb23A4S28an7ssH.conf
2025-01-29 13:13:37.178050: [debug] (SQL) SELECT "id","platform_ghc_dir","compiler","cabal_version","package_key","options_hash","haddock","library" FROM "precompiled_cache" WHERE "platform_ghc_dir"=? AND "compiler"=? AND "cabal_version"=? AND "package_key"=? AND "options_hash"=? AND "haddock"=?; [PersistText "x86_64-linux-nix/",PersistText "ghc-9.6.4",PersistText "3.10.1.0",PersistText "d4405abe08cf99c47f2af8534346492a5679e17a19529667b3eeddad4c0f0149,1184",PersistByteString "\239\146\&5\170\246\247\176\166!\172\237\244.S\US`\247\181\DC4\CAN\223\151\199v\250\DC3\189\f\172Q\152s",PersistBool False]
2025-01-29 13:13:37.178927: [debug] Reading package config:
/home/b/.stack/snapshots/x86_64-linux-nix/de1476877c5ac81108c78c024ecb0dc2066383ecf34f2d67504e0b8a1016f437/9.6.4/pkgdb/easy-file-0.2.5-HQl9n3284dh5L5XuBaANaK.conf
2025-01-29 13:13:37.179306: [debug] (SQL) SELECT "id", "precompiled_cache_id", "sub_lib" FROM "precompiled_cache_sub_lib" WHERE ("precompiled_cache_id"=?); [PersistInt64 7149]
2025-01-29 13:13:37.179487: [debug] (SQL) SELECT "id", "precompiled_cache_id", "exe" FROM "precompiled_cache_exe" WHERE ("precompiled_cache_id"=?); [PersistInt64 7149]
2025-01-29 13:13:37.182718: [info] ansi-terminal                    > using precompiled package

Stack version

stack --version
3.1.1 x86_64 hpack-0.37.0

Method of installation

NixOS package manager

Platform

NixOS 24.11.713790.2b4230bf03de (Vicuna)
@chreekat
Copy link
Member Author

sloww.txt

@mpilgrem
Copy link
Member

mpilgrem commented Feb 8, 2025

@chreekat, I don't know about speed, not least because Stack is likely doing things in different threads and that makes the interleaved debug output difficult to unpick, but I think Stack is behaving as expected, as follows:

  • by changing a (deemed) immutable extra-dep, you are specifying a new cache in the snapshot directory of the Stack root; and
  • Stack is working out what it can re-use from other existing caches and copying from those caches to the new cache. It announces it is doing that with using precompiled package.

@mpilgrem
Copy link
Member

mpilgrem commented Feb 8, 2025 •
edited
Loading

For example, building Stack itself, if I add acme-missles-0.3 as a (redundant) extra-dep, the build initially speeds though a number of using precompiled package until:

  • on Windows 11/x86_64, it hits open-browser-0.2.1.0 (which package, a direct dependency, for some reason, Stack decides it needs to build). While it is building that package in one thread, it continues to use some further using precompiled package (but a little slower than before). It then hits other packages Stack decides it needs to build: bitvec, indexed-traversable-instances, rio, serialise, text-metrics, th-lift-instances, hackage-security, semialign, witherable ... and things slow down; and
  • on macOS 14.2.1/AArch64, until it hits hackage-security (having gone past open-browser) and Stack also decides it needs to build bitvec, indexed-traversable-instances, rio, serialise, text-metrics, th-lift-instances, semialign, witherable ...

EDIT: I wonder what stops Stack from using a precompiled package in some cases. To investigate ...

@chreekat
Copy link
Member Author

Aha, I see. I was indeed naively assuming single-threaded output.

I'm not immediately convinced that accounts for everything, though. The dep I changed was pretty close to the top of the graph - it should not have evicted very many packages. I'm just waving my hands, though, because I haven't tried to line up lines of output with the package being processed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@chreekat @mpilgrem