ci: add rust-cache action

[Firestar99]

Maybe it'll speed up our CI?
https://github.com/Swatinem/rust-cache

My investigation log

Adding caching

First run (with no cache present) is looking real bad:

• windows test +1m 8s
• windows difftest +6m 24s
  • no idea why the cache would take so long when there's nothing to download and nothing has been compiled???
• all "install rust toolchain" are now 0s, so we probably have to move them before the cache action

First CI with restore

Comparing latest main to this PR:

• overall time: 27m 39s -> 16m 8s
• longest job "windows-difftest": 27m 29s -> 16m 0s
• cache restore, measured from seconded empty commit in this PR, which may have been removed:
  • windows: 58s to 2m 3s
    • first run of windows difftest has an obscene "restore" of 6m 24s even though there's nothing to restore???
  • linux: ~30s
  • mac: 30s - 40s
• cache creation, measured from first commit in this PR:
  • windows: 30s - 1m
  • linux: 8s - 15s
  • mac: 30s - 50s

Install toolchain before restore (run attempt #1)

• rust-cache seems to install the toolchain for you via rustup if you don't do it yourself, toolchain install has 0s execution time
• Action: move it before cache restore
• Results: old rust-cache time is now split between rustup install and new rust-cache
• Could caches contain rustup builds? Try purging caches

purging caches and retrying CI (run attempt #2)

• as expected, a lot of runtime is moved from "rust-cache" to "install toolchain"
• rust-cache restore times:
  • windows: 2-3s
  • mac: 1-2s
  • linux: 1-2s
  • => couldn't repo crazy CI times from first attempt, just a fluke?
• cache creation:
  • windows: 30-50s
  • mac: 30-40s, 60s fluke?
  • linux: ~10s
• Results: rustup before rust-cache to separate out toolchain install time

Lessen cache usage

• github cache is reporting high cache usage
  • 10GB limit by gh
  • caches take up:

job	windows	mac	linux	total
test	620 MB	590 MB	740 MB	1950 MB
compiletest	310 MB	300 MB	320 MB	930 MB
difftest	550 MB	550 MB	560 MB	1660 MB
lint			720 MB	720 MB
android			460 MB	460 MB
vulkan-sdk	290 MB	500 MB	460 MB	1250 MB
total	1770 MB	1940 MB	3260 MB	6970 MB

• Idea: can we reuse the same cache between actions?
  • only test creates the cache, for each platform
  • difftest may get a separate cache for the difftest target dir
• attempt #1, creating caches
  • windows randomly taking 15min to rustup install toolchain, likely the culprit of rust-cache taking so long the first time?
• attempt #2, restoring caches
  • Mac: vulkan sdk caching doesn't work at all, but still creates new caches? Cache restore on MacOS broken jakoch/install-vulkan-sdk-action#558
  • jobs have different cache hash due to rustup env vars... retry entire experiment

remove rustup env vars

• cache usage (limit of 10GB):

job	windows	mac	linux	total
test	620 MB	580 MB	730 MB	1930 MB
difftest	230 MB	240 MB	240 MB	710 MB
android			460 MB	460 MB
vulkan-sdk	290 MB	500 MB	460 MB	1250 MB
total	1140 MB	1320 MB	1890 MB	4350 MB

• CI times:
  • own cache @ de88135
  • test cache @ 7519e71
  • main @ 79acad1

job	windows	mac	linux
compiletest own cache	14m 17s	7m 43s	6m 12s
compiletest test cache	16m 4s	7m 28s	6m 21s
compiletest main	19m 32s	14m 42s	8m 3s
difftest own cache	15m 15s	10m 29s	8m 22s
difftest test cache	21m 18s	16m 11s	11m 8s
difftest main	26m 30s	15m 0s	14m 30s
lint own cache			1m 42s
lint test cache			6m 1s
lint main			5m 58s

• Analysis of difftest slowdown:
  • windows & mac +6min:
    • +3min from "test difftest" aka. cargo test -p difftest
    • +3min from "difftest" themselves
  • linux follows similar pattern but less worse

move "test difftest" to test job

• Idea: On main windows difftest:
  • test difftest: 7m 29s
  • difftest: 14m 44s
  • move work from the longest running task difftest to test
  • maybe also improve rust-cache reuse, and speed up difftest some more?

[nazar-pc]

In my projects I usually disable caching on Windows. Their gzip or whatever compression they use is so spectacularly slow that it is faster to download/install things from scratch than decompressing/recompressing cache.