We only need the compiler built for `yarn test` in the root directory.
Rather than always cache both for every step, let's just do it where
it's needed explicitly.
---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/facebook/react/pull/32608).
* #32609
* __->__ #32608