•   Blog   •   Contributions to GHC 9.2

Contributions to GHC 9.2
Tags: GHC, Haskell October 28, 2021

This is my GHC activities report for GHC 9.2.

I’ve made roughly 204 commits for the GHC 9.2 series at the time of writing.

This blog post only lists my personal contributions and thoughts and doesn’t engage my team nor IOG.

Constant-folding

fromIntegral is the go-to function to convert between integral types. It is implemented as follows:

It used to be a bit magical because it had a lot of associated rewrite rules to avoid passing through Integer when possible. For example, fromIntegral :: Int -> Word can be implemented much more efficiently by using the int2Word# primop (which is a no-op in assembly). This was done with a rewrite rule on fromIntegral.

Now, all the rewrite rules for fromIntegral are gone as the rewriting is done on Integer primitives themselves. For example, integerToWord# (integerFromInt# x) ~~> int2Word# x.

It is especially useful for alternatives Prelude that can now provide safer numeric conversion functions without relying on fromIntegral. Also as fromIntegral was (and still is) inlined to avoid function calls, we are more likely to catch constant-folding occuring after some inlining happened, as well as constant-folding occuring from direct use of toInteger/fromInteger or underlying primitives.

Note that Natural is also taken into account everytime I’m changing something Integer related to avoid differences between the two bignum types.

64-bit primops

GHC provides a few primitive types: Int#, Word# but also Int8#, Word32#, etc.. Word64# and Int64# were only used on 32-bit architectures (on 64-bit architectures Int#/Word# are 64-bit and can be used instead) but we are working (with John Ericson) on making them available on 64-bit architectures too. To make GHC a multi-target compiler, we want to avoid having primops whose type depend on the target so we want to use the explicitly sized Int64#/Word64# types for them.

At this point it is only teasing for the GHC 9.4 activities report. But in the process, in GHC 9.2 we have made all the operations on Int64#/Word64# proper primops instead of FFI calls. It allowed us to add constant-folding for these operations! There were some noticeable performance improvements on 32-bit architectures:

haddock.Cabal(normal) run/alloc 14215777340.8 12780374172.0 -10.1% GOOD

haddock.base(normal) run/alloc 15420020877.6 13643834480.0 -11.5% GOOD

Target constants

GHC doesn’t fully control code generation by the C toolchain used to build the runtime system (RTS). As such, GHC’s build system queries some structure sizes, field offsets and other values from object files generated by the C toolchain (this is the role of utils/deriveConstants program). These constants are then used when compiling Haskell code, e.g. the native code generator uses them to generate assembly.

These constants were stored in a globally installed platformConstants file. As we want to make GHC multi-target and the RTS reinstallable, it had to change. For example the reinstalled RTS may slightly change the structure representations (e.g. have packed fields, some field order changes, some alignment changes…). Hence platform constants are now only stored in the RTS unit, not globally.

Windows

At some point GHC’s Windows CI capacity became too small and it started slowing the merge process too much. So I’ve decided to rent a Windows machine and to install a Gitlab runner on it to build GHC…

I’ve discovered that Windows is a very hostile (Haskell) development platform. For example Git automatically converts LF line-ending characters into CRLF on checkout by default leading to weird build failures.

It seems I was the first one using a newer Cabal version which failed building GHC because a Windows-only utility (touchy) had a wrong cabal-version in its .cabal file.

More importantly it seems that I was also the first one building GHC on Windows with autotools 2.70 (first new version after 8 years) which broke the support of auxiliary files (e.g. config.guess) on Windows (https://gitlab.haskell.org/ghc/ghc/-/issues/19189). The handling of paths and path separators is a complete mess (see https://github.com/haskell/cabal/issues/7494 and https://github.com/haskell/cabal/issues/7649 where cabal devs try to fix the issue properly) so it was simpler to fix the issue by removing the use of config.guess and config.sub. We support 3.5 OSs and 4 architectures, why do we have to use these scripts anyway?

Modularity

A lot of time has been spent refactoring the compiler. I’ve started writing a paper/report about it so I won’t go into much details here.

Error as ADTs

I’ve kicked off the use of ADTs to represent errors and warnings so that they are easier to consume by other tools (instead of forcing them to parse text messages). This [first patch](https://gitlab.haskell.org/ghc/ghc/-/commit/a5aaceecaa) fixed the parser.

For some reason this patch improves performance:

Surprisingly, this patch enhances performance. On CI
(x86_64/deb9/hadrian, ghc/alloc):

   parsing001         -11.5%
   T13719             -2.7%
   MultiLayerModules  -3.5%
   Naperian           -3.1%

Appendix: commit list

# numeric

4d2bbd676e Remove ad-hoc fromIntegral rules
a24044dcb0 Ensure that newtype deriving strategy is used for CTypes
ab14cdbecb Fix naturalToFloat/Double
d1a9167606 Additional constant-folding rule for binary AND/OR
e552296414 Bignum: bump to version 1.1 (#19846)
1d7840f509 Detect underflow in fromIntegral/Int->Natural rule
882cc36382 Bignum: add BigNat Eq/Ord instances (#19647)
d197fb3d5c Bignum: remove unused extra files
3dc0815eea Bignum: fix invalid hs-boot declaration (#19638)
e594d067b2 Bignum: match on DataCon workers in rules (#19892)
773e2828fd Bignum: add Natural constant folding rules (#15821)
5e71dd3379 Bignum: fix bogus rewrite rule (#19345)
6f9a817fff Bignum: fix for Integer/Natural Ord instances
bff74de713 Bignum: make GMP's bignat_add not recursive
74ee1237bf Bignum: fix bigNatCompareWord# bug (#18813)
12191a99d3 Bignum: match on small Integer/Natural
8dd4f40512 Bignum: implement integerPowMod (#18427)
12c06927a0 Bignum: implement integerRecipMod (#18427)
e5523324d2 Bignum: add integerNegate RULE
1cde295c54 Bignum: add bigNatFromWordArray
04bc50b3c8 Bignum: implement extended GCD (#18427)
92daad241b Bignum: refactor backend modules
b689f3db02 Bignum: add clamping naturalToWord (fix #18697)
85e6212349 Bignum: add missing compat import/export functions
a8a2568b7b Bignum: add BigNat compat functions (#18613)
dc476a5040 Bignum: fix BigNat subtraction (#18604)
3f2f771869 Bignum: add more BigNat compat functions in integer-gmp
d613ed7624 Bignum: add backward compat integer-gmp functions
380638a336 Bignum: fix powMod for gmp backend (#18515)
6ee07b494d Bignum: add support for negative shifts (fix #18499)
96c31ea1f0 Fix bug in Natural multiplication (fix #18509)
0a3723876c Fix documentation and fix "check" bignum backend (#18604)
2754419673 Natural: fix left shift of 0 (fix #19170)
730ef38f46 Simplify constant-folding (#18032)
a98593f0c7 Refactor numeric constant folding rules
ffc3da474b Remove outdated note
85d899c8d3 Make proper fixed-width number literals

# cross-compilation / multi-target

52e5208367 Add Word64#/Int64# primops
ffb4762f2c Constants: add a note and fix minor doc glitches
201e29c74e Don't produce platformConstants file
0e9069be0e Read constants header instead of global platformConstants
ddd8fe23fd Generate parser for DerivedConstants.h
6c8702aede Remove dynamic-by-default (#16782)
a91dcb669b Don't get host RTS ways via settings (#18651)
accdb24a08 Expose RTS-only ways (#18651)
b695e7d736 Rename ghci flag into internal-interpreter
7721b923d5 Move GHC.Platform into the compiler
73145d57f9 Remove dead code in utils/derivConstants
9dfeca6c20 Remove platform constant wrappers
6333d73910 Put PlatformConstants into Platform
f7cc431341 Replace HscTarget with Backend
735f9d6bac Replace ghcWithNativeCodeGen with a proper Backend datatype
e079bb721e Correctly test active backend
8ea33edb2b Remove unused sGhcWithNativeCodeGen
bab4ec8f62 Don't panic if the NCG isn't built (it is always built)
0f17b930c1 Remove unused "ncg" flag
cdbd16f545 Fix toArgRep to support 64-bit reps on all systems
05c5c0549b Move loader state into Interp
d8dc0f9623 Fix array and cleanup conversion primops (#19026)
94bbc45df3 Use target Int/Word when detecting literal overflows (#17336)
2895fa6035 ghci: reuse Arch from ghc-boot
37f0434d65 Constant-folding: don't pass through GHC's Int/Word (fix #11704)

# Windows

dd23bd740f Windows: fix crlf on checkout
bcaa36c427 Fix Windows build with autoconf >=2.70 (#19189)
5b752b1d5c touchy: use a valid cabal-version
42ab06f793 Replace more autotools obsolete macros (#19189)
0f2891f018 configure: avoid empty lines in AC_CONFIG_FILES

# Modularity / Refactorings

d0e8c10d58 Move Unit related fields from DynFlags to HscEnv
76be0e32d6 Move SizedSeq into ghc-boot
690c894616 Parser: move parser utils into their own module
1ff61314fa Fix wrong comment about UnitState
c85f4928d4 Refactor -dynamic-too handling
72f2257c79 Don't initialize plugins in the Core2Core pipeline
ecfd0278cb Move Plugins into HscEnv (#17957)
14ce454f72 Linker: reorganize linker related code
08e6993a1b Move loadDecl into IfaceToCore
0e9f6defbd Split GHC.Driver.Types
9648d680b4 Remove pdocPrec
0a5f291859 Parser: don't require the HomeUnitId
9bbc84d20d DynFlags: refactor DmdAnal
6a243e9daa Cache HomeUnit in HscEnv (#17957)
386c2d7ff8 Use UnitId in the backend instead of Unit
93d5de165a Don't import GHC.Unit to reduce the number of dependencies
dafe79433c Parser: remove some unused imports
a946c7ef9e Less DynFlags in Header parsing
a5aaceecaa Use ADTs for parser errors/warnings
9befd94d79 Remove unused global variables
df3f588075 Remove unsafeGlobalDynFlags (#17957, #14597)
8e3f00dd24 Make the parser module less dependent on DynFlags
a997fa01d9 Preliminary work towards removing DynFlags -> Driver.Ppr dependency
a584366b1d Remove sdocWithDynFlags (fix #10143)
667d63558a Refactor CLabel pretty-printing
7f2785f2d6 Remove pprPrec from Outputable (unused)
37aa224ad5 Add note about OutputableP
e45c85446d Generalize OutputableP
ca48076ae8 Introduce OutputableP
8af954d202 Make ghc-boot reexport modules from ghc-boot-th
ffae579211 Add comments about sm_dflags and simpleOptExpr
b3df72a699 DynFlags: add sm_pre_inline field into SimplMode (#17957)
3f32a9c0f4 DynFlags: add UnfoldingOpts and SimpleOpts
8c89268905 DynFlags: add OptCoercionOpts
df04b81e12 Move DynFlags test into updateModDetailsIdInfos's caller (#17957)
220ad8d67a DynFlags: don't pass DynFlags to cmmImplementSwitchPlans
89ce7cdf97 DynFlags: use Platform in foldRegs*
1d6d648866 Don't rely on CLabel's Outputable instance in CmmToC
bcb68a3f7f Don't store HomeUnit in UnitConfig
0b17fa185a Refactor UnitId pretty-printing
659eb31b7a NCG: Dwarf configuration
50eb4460cd Don't use DynFlags in CmmToAsm.BlockLayout (#17957)
293c7fba6c Put CFG weights into their own module (#17957)
0c5ed5c7eb DynFlags: refactor GHC.CmmToAsm (#17957, #10143)
accbc242e5 DynFlags: disentangle Outputable
6334815540 Use a type alias for Ways
3b15dc3cfb DynFlags: don't use sdocWithDynFlags in GHC.CmmToAsm.Dwarf.Types
56a7c19337 Refactor CLabel pretty-printing
175cb5b404 DynFlags: don't use sdocWithDynFlags in datacon ppr
342a01af62 Add GHC.Platform.Profile
15b2b44fe3 Rename GHC.Driver.Ways into GHC.Platform.Ways
e443846ba0 DynFlags: store printer in TraceBinIfaceReading
7f44df1ec6 Minor refactoring of Unit display
6e2db34bdf Add accessors to ArchOS
ffc0d578ea Add HomeUnit type
8e2f85f6b4 Refactor Logger
e9d0dd4554 Move showSDocUnsafe/showPprUnsafe back into GHC.Utils.Outputable
daa6363f49 DynFlags: move temp file management into HscEnv (#17957)
3aceea907f Don't pass homeUnitId at ExternalPackageState creation time (#10827)
957b53760e Core: introduce Alt/AnnAlt/IfaceAlt datatypes
fd0945b7bf Move Hooks into HscEnv

# performance

e393f213f5 Allow fusion with catMaybes (#18574)
05550a5abc Avoid roundtrip through SDoc
b1eb38a0a7 Perf: make SDoc monad one-shot (#18202)
8a51b2ab74 Make IOEnv monad one-shot (#18202)
bee43aca82 Rewrite and move the monad-state hack note
77398b678a Avoid allocations in `splitAtList` (#18535)
acf537f9fe Make splitAtList strict in its arguments
fe4202cef7 Always INLINE ($!)
b677d09344 Make sized division primops ok-for-spec (#19026)
87ae062ab8 Compute length only once in foldBal
d412cd108b Write explicit IOEnv's Functor and MonadIO instances (#18202)
8a433a3c88 Fix leaks of the HscEnv with quick flavour (#19356)
72c0e07869 Make known names simple ConApps (#19386)
887eb6ec23 Enhance Data instance generation
ae8379abb8 Ppr: compute length of string literals at compile time (#19266)
0249974e76 Fix strictness in TyCo.Tidy (#14738)
5ae73f69e5 Add regression test for #16577
6c771aafa3 Implement Unique supply with Addr# atomic primop
4af6126d17 Use static array in zeroCount

# build system / CI

a3129aa743 Hadrian: fix .cabal file
31ee48dc64 CI: reduce xz compression for non release/nightly jobs
18e106a8df Add missing .hi-boot dependencies with ghc -M (#14482)
9c87f97ee9 Fix spurious failures of T16916 on CI (#16966)
e516ef7eb7 Hadrian: fix flavour parser
bd877edd94 Hadrian: show default ghc-bignum backend (fix #18912)
659fcb1493 Fix project version for ProjectVersionMunged (fix #19058)
7cb5df9617 hadrian: fix ghc-pkg uses (#17601)
75fc1ed58b Hadrian: fix detection of ghc-pkg for cross-compilers
66414bdf40 configure: fix the use of some obsolete macros (#19189)
4813486f87 Move Hadrian's wiki pages in tree (fix #16165)
3486ebe6f9 Hadrian: don't fail if ghc-tarballs dir doesn't exist
b4a929a1e5 Hadrian: fix libffi tarball parsing
aaa8f00fa0 Validate script: fix configure command when using stack
db236ffc03 Testsuite: increase timeout for T18223 (#18795)
3a16d764f3 ghci: don't compile unneeded modules
e691a5a046 Hadrian: add quick-debug flavour
f1accd0096 Add quick-validate Hadrian flavour (quick + -Werror)
2da933084b Hadrian: fix slow-validate flavour (#18586)
459afeb592 Fix build systems
b3143f5a08 Enhance metrics output
9c1b8ad931 Bump Stack resolver
a01e78cc5b Don't build extra object with -no-hs-main
226cefd0a8 Fix fake import in GHC.Exception.Type boot module

# misc

644424afc5 GHCi: don't discard plugins on reload (#20335)
3a81545208 RTS: try to fix timer races
6b10163e0d Disable bogus assertion (#19489)
d14a20686c Enhance pass result forcing
3f9af89128 Add a flag to dump the FastString table
38748d5fb4 Minor simplification for leak indicators
fe78997845 IntVar: fix allocation size
a3d995fa18 Fix -dynamic-too with wired-in modules (#19264)
13d876bab3 Enhance nested TransCo pretty-printing
db16302cfd LLVM: fix sized shift primops (#19215)
29173f8888 Factorize and document binder collect functions
01ea56a22d Arrows: collect evidence binders
84dcb8440e Revert "Remove SpecConstrAnnotation (#13681)" (#19168)
fe344da9be Missing fields: enhance error messages (#18869)
d930687a07 Show missing field types (#18869)
f59c34b8e4 Support package qualifier in Prelude import
480a38d4ad rts: don't use siginterrupt (#19019)
f9f9f030d7 Arrows: correctly query arrow methods (#17423)
381eb66012 Display FFI labels (fix #18539)
52114fa0f9 Add Addr# atomic primops (#17751)
4c407f6e71 Export SPEC from GHC.Exts (#13681)
81560981fd Don't use LEA with 8-bit registers (#18614)
22f5d9a951 GC: Avoid data race (#18717, #17964)
809f09e8a7 Fix parsing of PIE flags
0e8b923d3c Apply suggestion to compiler/GHC/SysTools.hs
74d4017b74 Fix -flink-rts (#18651)
0bb02873ee Add test for T18574
4b4fbc58d3 Remove "Ord FastString" instance
884245dd29 Fix FastString lexicographic ordering (fix #18562)
2d8ca91703 Fix -ddump-stg flag
7c274cd530 Fix minimal imports dump for boot files (fix #18497)
9e2930c3e1 Bump CountParserDeps
ff1b7710c9 Add test for #18064