Cherrypick #29811
When we made stylesheets suspend even during high priority updates we
exposed a bug in the loading tracking of stylesheets that are loaded as
part of the preamble. This allowed these stylesheets to put suspense
boundaries into fallback mode more often than expected because cases
where a stylesheet was server rendered could now cause a fallback to
trigger which was never intended to happen.
This fix updates resource construction to evaluate whether the instance
exists in the DOM prior to construction and if so marks the resource as
loaded and inserted.
One ambiguity that needed to be solved still is how to tell whether a
stylesheet rendered as part of a late Suspense boundary reveal is
already loaded. I updated the instruction to clear out the loading
promise after successfully loading. This is useful because later if we
encounter this same resource again we can avoid the microtask if it is
already loaded. It also means that we can concretely understand that if
a stylesheet is in the DOM without this marker then it must have loaded
(or errored) already.
This lets us ensure that we use the original V8 format and it lets us
skip source mapping. Source mapping every call can be expensive since we
do it eagerly for server components even if an error doesn't happen.
In the case of an error being thrown we don't actually always do this in
practice because if a try/catch before us touches it or if something in
onError touches it (which the default console.error does), it has
already been initialized. So we have to be resilient to thrown errors
having other formats.
These are not as perf sensitive since something actually threw but if
you want better perf in these cases, you can simply do something like
`onError(error) { console.error(error.message) }` instead.
The server has to be aware whether it's looking up original or compiled
output. I currently use the file:// check to determine if it's referring
to a source mapped file or compiled file in the fixture. A bundled app
can more easily check if it's a bundle or not.
Normally we take the renderClientElement path but this is an internal
fast path.
No tests because we don't run tests with console.createTask (which is
not easy since we test component stacks).
Ideally this would be covered by types but since the types don't
consider flags and DEV it doesn't really help.
## Overview
We didn't have any tests that ran in persistent mode with the xplat
feature flags (for either variant).
As a result, invalid test gating like in
https://github.com/facebook/react/pull/29664 were not caught.
This PR adds test flavors for `ReactFeatureFlag-native-fb.js` in both
variants.
<!--
Thanks for submitting a pull request!
We appreciate you spending the time to work on these changes. Please
provide enough information so that others can review your pull request.
The three fields below are mandatory.
Before submitting a pull request, please make sure the following is
done:
1. Fork [the repository](https://github.com/facebook/react) and create
your branch from `main`.
2. Run `yarn` in the repository root.
3. If you've fixed a bug or added code that should be tested, add tests!
4. Ensure the test suite passes (`yarn test`). Tip: `yarn test --watch
TestName` is helpful in development.
5. Run `yarn test --prod` to test in the production environment. It
supports the same options as `yarn test`.
6. If you need a debugger, run `yarn test --debug --watch TestName`,
open `chrome://inspect`, and press "Inspect".
7. Format your code with
[prettier](https://github.com/prettier/prettier) (`yarn prettier`).
8. Make sure your code lints (`yarn lint`). Tip: `yarn linc` to only
check changed files.
9. Run the [Flow](https://flowtype.org/) type checks (`yarn flow`).
10. If you haven't already, complete the CLA.
Learn more about contributing:
https://reactjs.org/docs/how-to-contribute.html
-->
## Summary
Remove `startTransition` and `useActionState` from `react-server`
condition of react, as they should only stay in client bundle.
This will reduce the server bundle of react itself.
Found this while tracing where the `process.emit` was called.
<!--
Explain the **motivation** for making this change. What existing problem
does the pull request solve?
-->
## How did you test this change?
<!--
Demonstrate the code is solid. Example: The exact commands you ran and
their output, screenshots / videos if the pull request changes the user
interface.
How exactly did you verify that your PR solves the issue you wanted to
solve?
If you leave this empty, your PR will very likely be closed.
-->
Following the instructions in the compiler/docs/DEVELOPMENT_GUIDE.md, we are stuck on the command `yarn snap --watch` because it calls readTestFilter even though the filter option is not enabled.
Use some clever git diffing to ignore lines that only change the
`@generated` header. We can't do this for the version string because the
version string can be embedded in lines with other changes, but this
header is always on one line.
This information is available in the regular stack but since that's
hidden behind an expando and our appended stack to logs is not hidden,
it hides the most important frames like the name of the current
component.
This is closer to what happens to the native stack.
We only include stacks if they're within a ReactFiberCallUserSpace call
frame. This should be most that have a current fiber but this is
critical to filtering out most React frames if the regular node_modules
filter doesn't work.
Most React warnings fire during the rendering phase and not inside a
user space function but some do like hooks warnings and setState in
render. This feature is more important if we port this to React DevTools
appending stacks to all logs where it's likely to originate from inside
a component and you want the line within that component to immediately
part of the visible stack.
One thing that kind sucks is that we don't have a reliable way to
exclude React internal stack frames. We filter node_modules but it might
not match. For other cases I try hard to only track the stack frame at
the root of React (e.g. immediately inside createElement) until the
ReactFiberCallUserSpace so we don't need the filtering to work. In this
case it's hard to achieve the same thing though. This is easier in RDT
because we have the start/end line and parsing of stack traces so we can
use that to exclude internals but that's a lot of code/complexity for
shipping within the library.
For example in Safari:
<img width="590" alt="Screenshot 2024-05-31 at 6 15 27 PM"
src="https://github.com/facebook/react/assets/63648/2820c8c0-8a03-42e9-8678-8348f66b051a">
Ideally warnOnUseFormStateInDev and useFormState wouldn't be included
since they're React internals. Before this change, the Counter.js line
also wasn't included though which points to exactly where the error is
within the user code.
(Note Server Components have V8 formatted lines and Client Components
have JSC formatted lines.)
RC releases are a special kind of prerelease build because unlike
canaries we shouldn't publish new RCs from any commit on `main`, only
when we intentionally bump the RC number. But they are still prerelases
— like canary and experimental releases, they should use exact version
numbers in their dependencies (no ^).
We only need to generate these builds during the RC phase, i.e. when the
canary channel label is set to "rc".
Example of resulting package.json output:
```json
{
"name": "react-dom",
"version": "19.0.0-rc.0",
"dependencies": {
"scheduler": "0.25.0-rc.0"
},
"peerDependencies": {
"react": "19.0.0-rc.0"
}
}
```
https://react-builds.vercel.app/prs/29736/files/oss-stable-rc/react-dom/package.json
Based on
- #29694
---
If an action in the useActionState queue errors, we shouldn't run any
subsequent actions. The contract of useActionState is that the actions
run in sequence, and that one action can assume that all previous
actions have completed successfully.
For example, in a shopping cart UI, you might dispatch an "Add to cart"
action followed by a "Checkout" action. If the "Add to cart" action
errors, the "Checkout" action should not run.
An implication of this change is that once useActionState falls into an
error state, the only way to recover is to reset the component tree,
i.e. by unmounting and remounting. The way to customize the error
handling behavior is to wrap the action body in a try/catch.
Mini-refactor of useActionState to only wrap the action in a transition
context if the dispatch is called during a transition. Conceptually, the
action starts as soon as the dispatch is called, even if the action is
queued until earlier ones finish.
We will also warn if an async action is dispatched outside of a
transition, since that is almost certainly a mistake. Ideally we would
automatically upgrade these to a transition, but we don't have a great
way to tell if the action is async until after it's already run.
Host Components can exist as four semantic types
1. regular Components (Vanilla obv)
2. singleton Components
2. hoistable components
3. resources
Each of these component types have their own rules related to mounting
and reconciliation however they are not direclty modeled as their own
unique fiber type. This is partly for code size but also because
reconciling the inner type of these components would be in a very hot
path in fiber creation and reconciliation and it's just not practical to
do this logic check here.
Right now we have three Fiber types used to implement these 4 concepts
but we probably need to reconsider the model and think of Host
Components as a single fiber type with an inner implementation. Once we
do this we can regularize things like transitioning between a resource
and a regular component or a singleton and a hoistable instance. The
cases where these transitions happen today aren't particularly common
but they can be observed and currently the handling of these transitions
is incomplete at best and buggy at worst. The most egregious case is the
link type. This can be a regular component (stylesheet without
precedence) a hoistable component (non stylesheet link tags) or a
resource (stylesheet with a precedence) and if you have a single jsx
slot that tries to reconcile transitions between these types it just
doesn't work well.
This commit adds an error for when a Hoistable goes from Instance to
Resource. Currently this is only possible for `<link>` elements going to
and from stylesheets with precedence. Hopefully we'll be able to remove
this error and implement as an inner type before we encounter new
categories for the Hoistable types
detecting type shifting to and from regular components is harder to do
efficiently because we don't want to reevaluate the type on every update
for host components which is currently not required and would add
overhead to a very hot path
singletons can't really type shift in their one practical implementation
(DOM) so they are only a problem in theroy not practice
Requires https://github.com/facebook/react/pull/29706
The strategy here is to:
- Checkout the builds/facebook-www branch
- Read the current sync'd VERSION
- Checkout out main and sync new build
- sed/{new version string}/{old version string}
- Run git status, skip sync if clean
- Otherwise, sed/{old version string}/{new version string} and push
commit
This means that:
- We're using the real version strings from the builds
- We are checking the last commit on the branch for the real last
version
- We're skipping any commits that won't result in changes
- ???
- Profit!
This lets you click a stack frame on the client and see the Server
source code inline.
<img width="871" alt="Screenshot 2024-06-01 at 11 44 24 PM"
src="https://github.com/facebook/react/assets/63648/581281ce-0dce-40c0-a084-4a6d53ba1682">
<img width="840" alt="Screenshot 2024-06-01 at 11 43 37 PM"
src="https://github.com/facebook/react/assets/63648/00dc77af-07c1-4389-9ae0-cf1f45199efb">
We could do some logic on the server that sends a source map url for
every stack frame in the RSC payload. That would make the client
potentially config free. However regardless we need the config to
describe what url scheme to use since that’s not built in to the bundler
config. In practice you likely have a common pattern for your source
maps so no need to send data over and over when we can just have a
simple function configured on the client.
The server must return a source map, even if the file is not actually
compiled since the fake file is still compiled.
The source mapping strategy can be one of two models depending on if the
server’s stack traces (`new Error().stack`) are source mapped back to
the original (`—enable-source-maps`) or represents the location in
compiled code (like in the browser).
If it represents the location in compiled code it’s actually easier. You
just serve the source map generated for that file by the tooling.
If it is already source mapped it has to generate a source map where
everything points to the same location (as if not compiled) ideally with
a segment per logical ast node.
Eslint rules should never throw, so if we fail to parse with Babel or
Hermes, we should just ignore the error. This should fix issues such as
trying to run the eslint rule on non tsx|ts|jsx|js files, Hermes parser
not supporting certain JS syntax, etc.
I didn't add a test for this as our eslint-rule-tester config uses
hermes-eslint parser, so it wasn't possible to add a top level await as
it would crash hermes-eslint before our rule was triggered. Similarly I
couldn't add a test for non-JS files as it would not be parseable by
hermes-eslint.
Fixes#29107
ghstack-source-id: 60afcdb89ab4a8d2e4697cc50c5490803e7cbeac
Pull Request resolved: https://github.com/facebook/react/pull/29631
When a component suspends with `use`, we switch to the "re-render"
dispatcher during the subsequent render attempt, so that we can reuse
the work from the initial attempt. However, once we run out of hooks
from the previous attempt, we should switch back to the regular "update"
dispatcher.
This is conceptually the same fix as the one introduced in
https://github.com/facebook/react/pull/26232. That fix only accounted
for initial mount, but the useTransition regression test added in
f82973302b3f490ec120c3b102e8c3792452dfc9 illustrates that we need to
handle updates, too.
The issue affects more than just useTransition but because most of the
behavior between the "re-render" and "update" dispatchers is the same
it's hard to contrive other scenarios in a test, which is probably why
it took so long for someone to notice.
Closes#28923 and #29209
---------
Co-authored-by: eps1lon <sebastian.silbermann@vercel.com>
Summary: Using the change detection code to debug codebases that violate the rules of react is a lot easier when we have a source location corresponding to the value that has changed inappropriately. I didn't see an easy way to track that information in the existing data structures at the point of codegen, so this PR adds locations to identifiers and reactive scopes (the location of a reactive scope is the range of the locations of its included identifiers).
I'm interested if there's a better way to do this that I missed!
ghstack-source-id: aed5f7edda
Pull Request resolved: https://github.com/facebook/react/pull/29658
Summary: This PR expands the analysis from the previous in the stack in order to also capture when a value can incorrectly change within a single render, rather than just changing between two renders. In the case where dependencies have changed and so a new value is being computed, we now compute the value twice and compare the results. This would, for example, catch when we call Math.random() in render.
The generated code is a little convoluted, because we don't want to have to traverse the generated code and substitute variable names with new ones. Instead, we save the initial value to the cache as normal, then run the computation block again and compare the resulting values to the cached ones. Then, to make sure that the cached values are identical to the computed ones, we reassign the cached values into the output variables.
ghstack-source-id: d0f11a4cb2
Pull Request resolved: https://github.com/facebook/react/pull/29657
Summary: jmbrown215 recently had an observation that the arguments to useState/useRef are only used when a component renders for the first time, and never afterwards. We can skip more computation that we previously could, with reactive blocks that previously recomputed values when inputs changed now only ever computing them on the first render.
ghstack-source-id: 5d044ef787
Pull Request resolved: https://github.com/facebook/react/pull/29653
Summary: The essential assumption of the compiler is that if the inputs to a computation have not changed, then the output should not change either--computation that the compiler optimizes is idempotent.
This is, of course, known to be false in practice, because this property rests on requirements (the Rules of React) that are loosely enforced at best. When rolling out the compiler to a codebase that might have rules of react violations, how should developers debug any issues that arise?
This diff attempts one approach to that: when the option is set, rather than simply skipping computation when dependencies haven't changed, we will *still perform the computation*, but will then use a runtime function to compare the original value and the resultant value. The runtime function can be customized, but the idea is that it will perform a structural equality check on the values, and if the values aren't structurally equal, we can report an error, including information about what file and what variable was to blame.
This assists in debugging by narrowing down what specific computation is responsible for a difference in behavior between the uncompiled code and the program after compilation.
ghstack-source-id: 50dad3dacf
Pull Request resolved: https://github.com/facebook/react/pull/29656
Summary: This adds a debugging mode to the compiler that simply adds a `|| true` to the guard on all memoization blocks, which results in the generated code never using memoized values and always recomputing them. This is designed as a validation tool for the compiler's correctness--every program *should* behave exactly the same with this option enabled as it would with it disabled, and so any difference in behavior should be investigated as either a compiler bug or a pipeline issue.
(We add `|| true` rather than dropping the conditional block entirely because we still want to exercise the guard tests, in case the guards themselves are the source of an error, like reading a property from undefined in a guard.)
ghstack-source-id: 955a47ec16
Pull Request resolved: https://github.com/facebook/react/pull/29655
Summary: This adds a compiler option to not drop existing manual memoization and leaving useMemo/useCallback in the generated source. Why do we need this, given that we also have options to validate or ensure that existing memoization is preserved? It's because later diffs on this stack are designed to alter the behavior of the memoization that the compiler emits, in order to detect rules of react violations and debug issues. We don't want to change the behavior of user-level memoization, however, since doing so would be altering the semantics of the user's program in an unacceptable way.
ghstack-source-id: 89dccdec9ccb4306b16e849e9fa2170bb5dd021f
Pull Request resolved: https://github.com/facebook/react/pull/29654
This lets any element created from the server, to bottom out with a
client "owner" which is the creator of the Flight request. This could be
a Server Action being invoked or a router.
This is similar to how a client element bottoms out in the creator of
the root element without an owner. E.g. where the root app element was
created.
Without this, we inherit the task of whatever is currently executing
when we're parsing which can be misleading.
Before:
<img width="507" alt="Screenshot 2024-05-30 at 12 06 57 PM"
src="https://github.com/facebook/react/assets/63648/e234db7e-67f7-404c-958a-5c5500ffdf1f">
After:
<img width="555" alt="Screenshot 2024-05-30 at 4 59 04 PM"
src="https://github.com/facebook/react/assets/63648/8ba6acb4-2ffd-49d4-bd44-08228ad4200e">
The before/after doesn't show much of a difference here but that's just
because our Flight parsing loop is an async, which maybe it shouldn't be
because it can be unnecessarily deep, and it creates a hidden line for
every loop. That's what the `Promise.then` is. If the element is lazily
initialized it's worse because we can end up in an unrelated render task
as the owner - although that's its own problem.
We're getting a ton of issues filed using the blank template, for
example these airline support tickets:
https://github.com/facebook/react/issues/29678
I think someone somewhere is linking to our issues with pre-filled
content. This fixes it by forcing a template to be used.
When defining a displayName on forwardRef/memo we forward that name to
the inner function.
We used to use displayName for this but in #29206 I switched this to use
`"name"`. That's because V8 doesn't use displayName, it only uses the
overridden name in stack traces. This is the only thing covered by our
tests for component stacks.
However, I realized that Safari only uses displayName and not the name.
So this sets both.
## Summary
In https://github.com/facebook/react/pull/29088, the validation logic
for `React.Children` inspected whether `mappedChild` — the return value
of the map callback — has a valid `key`. However, this deviates from
existing behavior which only warns if the original `child` is missing a
required `key`.
This fixes false positive `key` validation warnings when using
`React.Children`, by validating the original `child` instead of
`mappedChild`.
This is a more general fix that expands upon my previous fix in
https://github.com/facebook/react/pull/29662.
## How did you test this change?
```
$ yarn test ReactChildren-test.js
```
Follow up to https://github.com/facebook/react/pull/29632.
It's possible for `eval` to throw such as if we're in a CSP environment.
This is non-essential debug information. We can still proceed to create
a fake stack entry. It'll still have the right name. It just won't have
the right line/col number nor source url/source map. It might also be
ignored listed since it's inside Flight.
We have three kinds of stacks that we send in the RSC protocol:
- The stack trace where a replayed `console.log` was called on the
server.
- The JSX callsite that created a Server Component which then later
called another component.
- The JSX callsite that created a Host or Client Component.
These stack frames disappear in native stacks on the client since
they're executed on the server. This evals a fake file which only has
one call in it on the same line/column as the server. Then we call
through these fake modules to "replay" the callstack. We then replay the
`console.log` within this stack, or call `console.createTask` in this
stack to recreate the stack.
The main concern with this approach is the performance. It adds
significant cost to create all these eval:ed functions but it should
eventually balance out.
This doesn't yet apply source maps to these. With source maps it'll be
able to show the server source code when clicking the links.
I don't love how these appear.
- Because we haven't yet initialized the client module we don't have the
name of the client component we're about to render yet which leads to
the `<...>` task name.
- The `(async)` suffix Chrome adds is still a problem.
- The VMxxxx prefix is used to disambiguate which is noisy. Might be
helped by source maps.
- The continuation of the async stacks end up rooted somewhere in the
bootstrapping of the app. This might be ok when the bootstrapping ends
up ignore listed but it's kind of a problem that you can't clear the
async stack.
<img width="927" alt="Screenshot 2024-05-28 at 11 58 56 PM"
src="https://github.com/facebook/react/assets/63648/1c9d32ce-e671-47c8-9d18-9fab3bffabd0">
<img width="431" alt="Screenshot 2024-05-28 at 11 58 07 PM"
src="https://github.com/facebook/react/assets/63648/52f57518-bbed-400e-952d-6650835ac6b6">
<img width="327" alt="Screenshot 2024-05-28 at 11 58 31 PM"
src="https://github.com/facebook/react/assets/63648/d311a639-79a1-457f-9a46-4f3298d07e65">
<img width="817" alt="Screenshot 2024-05-28 at 11 59 12 PM"
src="https://github.com/facebook/react/assets/63648/3aefd356-acf4-4daa-bdbf-b8c8345f6d4b">
Partially reverts https://github.com/facebook/react/pull/28593.
While rolling out RDT 5.2.0, I've observed some issues on React Native
side: hooks inspection for some complex hook trees, like in
AnimatedView, were broken. After some debugging, I've noticed a
difference between what is in frame's source.
The difference is in the top-most frame, where with V8 it will correctly
pick up the `Type` as `Proxy` in `hookStack`, but for Hermes it will be
`Object`. This means that for React Native this top most frame is
skipped, since sources are identical.
Here I am reverting back to the previous logic, where we check each
frame if its a part of the wrapper, but also updated `isReactWrapper`
function to have an explicit case for `useFormStatus` support.
## Summary
https://github.com/facebook/react/pull/29088 introduced a regression
triggering this warning when rendering flattened positional children:
> Each child in a list should have a unique "key" prop.
The specific scenario that triggers this is when rendering multiple
positional children (which do not require unique `key` props) after
flattening them with one of the `React.Children` utilities (e.g.
`React.Children.toArray`).
The refactored logic in `React.Children` incorrectly drops the
`element._store.validated` property in `__DEV__`. This diff fixes the
bug and introduces a unit test to prevent future regressions.
## How did you test this change?
```
$ yarn test ReactChildren-test.js
```
## Summary
This PR add tests for `ReactNativeAttributePayloadFabric.js`.
It introduces `ReactNativeAttributePayloadFabric-test.internal.js`,
which is a copy-paste of `ReactNativeAttributePayload-test.internal.js`.
On top of that, there is a bunch of new test cases for the
`ReactNativeAttributePayloadFabric.create` function.
## How did you test this change?
```
yarn test packages/react-native-renderer
```
## Summary
Resolves#29622
## How did you test this change?
I verified the implementation using the test.
Note:
This PR was done without waiting for approval in #29622, so feel free to
just close it.
We were missing a check that ObjectMethods are not getters or setters. In our experience this is pretty rare within React components and hooks themselves, so let's start with a todo.
Closes#29586
ghstack-source-id: 03c6cce9a9
Pull Request resolved: https://github.com/facebook/react/pull/29592
Fixes https://x.com/raibima/status/1794395807216738792
The issue is that if you pass a global-modifying function as prop to JSX, we currently report that it's invalid to modify a global during rendering. The problem is that we don't really know when/if the child component will actually call that function prop. It would be against the rules to call the function during render, but it's totally fine to call it during an event handler or from a useEffect.
Since we don't know at the call-site how the child will use the function, we should allow such calls. In the future we could improve this in a few ways:
* For all functions that modify globals, codegen an assertion or warning into the function that fires if it's called "during render". We'd have to precisely define what "during render" is, but this would at least help developers catch this dynamically.
* Use the type system to distinguish "event/effect" and "render" functions to help developers avoid accidentally mutating globals during render.
ghstack-source-id: 4aba4e6d21
Pull Request resolved: https://github.com/facebook/react/pull/29591
- Moves the file as it needs to be in root git directory
- Removes now unreachable commits due to repo merge
- Add run prettier commit c998bb1ed4 to ignored revs
ghstack-source-id: d9dfa7099fbc7782fbce600af4caafd405c196cb
Pull Request resolved: https://github.com/facebook/react/pull/29630
This was missed in https://github.com/facebook/react/pull/29038 when
unifying the "owner" abstractions, causing `findNodeHandle` to warn even
outside of `render()` invocations.
user's pipeline
When the user app has a babel.config file that is missing the compiler,
strange things happen as babel does some strange merging of options from
the user's config and in various callsites like in our eslint rule and
healthcheck script. To minimize odd behavior, we default to not reading
the user's babel.config
Fixes#29135
ghstack-source-id: d6fdc43c5c
Pull Request resolved: https://github.com/facebook/react/pull/29211
After this is merged, I'll add it to .git-blame-ignore-revs. I can't do
it now as the hash will change after ghstack lands this stack.
ghstack-source-id: 054ca869b7
Pull Request resolved: https://github.com/facebook/react/pull/29214
This didn't actually fail before but I'm just adding an extra check.
Currently Client References are always "function" proxies so they never
fall into this branch. However, we do in theory support objects as
client references too depending on environment. We have checks
elsewhere. So this just makes that consistent.
When I added new builtin types for jsx and functions, i forget to add a shape definition. This meant that attempting to accesss a property or method on these types would cause an internal error with an unresolved shape. That wasn't obvious because we rarely call methods on these types.
I confirmed that the new fixtures here fail without the fix.
ghstack-source-id: aa8f8d75a3
Pull Request resolved: https://github.com/facebook/react/pull/29624
We currently don't report an error if the code attempts to reassign a const. Our thinking has been that we're not trying to catch all possible mistakes you could make in JavaScript — that's what ESLint, TypeScript, and Flow are for — and that we want to focus on React errors. However, accidentally reassigning a const is easy to catch and doesn't get in the way of other analysis so let's implement it.
Note that React Compiler's ESLint plugin won't report these errors by default, but they will show up in playground.
Fixes#29598
ghstack-source-id: a0af8b9a486d74a8991413322efddc3e3028c755
Pull Request resolved: https://github.com/facebook/react/pull/29619
Fixes the behavior of actions that are queued by useActionState to use
the action function that was current at the time it was dispatched, not
at the time it eventually executes.
The conceptual model is that the action is immediately dispatched, as if
it were sent to a remote server/worker. It's the remote worker that
maintains the queue, not the client.
This is another property of actions makes them more like event handlers
than like reducers.
`disableDOMTestUtils` and the FB build `ReactTestUtilsFB` allowed us to
finish migrating internal callsites off of ReactTestUtils. Now that
usage is cleaned up, we can remove the flag, build artifact, and test
coverage for the deprecated utility methods.
Throw an error during module initialization if the version of the
"react-dom" package does not match the version of "react".
We used to be more relaxed about this, because the "react" package
changed so infrequently. However, we now have many more features that
rely on an internal protocol between the two packages, including Hooks,
Float, and the compiler runtime. So it's important that both packages
are versioned in lockstep.
Before this change, a version mismatch would often result in a cryptic
internal error with no indication of the root cause.
Instead, we will now compare the versions during module initialization
and immediately throw an error to catch mistakes as early as possible
and provide a clear error message.
Stacked on #29206 and #29221.
This disables appending owner stacks to console when
`console.createTask` is available in the environment. Instead we rely on
native "async" stacks that end up looking like this with source maps and
ignore list enabled.
<img width="673" alt="Screenshot 2024-05-22 at 4 00 27 PM"
src="https://github.com/facebook/react/assets/63648/5313ed53-b298-4386-8f76-8eb85bdfbbc7">
Unfortunately Chrome requires a string name for each async stack and,
worse, a suffix of `(async)` is automatically added which is very
confusing since it seems like it might be an async component or
something which it is not.
In this case it's not so bad because it's nice to refer to the host
component which otherwise doesn't have a stack frame since it's
internal. However, if there were more owners here there would also be a
`<Counter> (async)` which ends up being kind of duplicative.
If the Chrome DevTools is not open from the start of the app, then
`console.createTask` is disabled and so you lose the stack for those
errors (or those parents if the devtools is opened later). Unlike our
appended ones that are always added. That's unfortunate and likely to be
a bit of a DX issue but it's also nice that it saves on perf in DEV mode
for those cases. Framework dialogs can still surface the stack since we
also track it in user space in parallel.
This currently doesn't track Server Components yet. We need a more
clever hack for that part in a follow up.
I think I probably need to also add something to React DevTools to
disable its stacks for this case too. Since it looks for stacks in the
console.error and adds a stack otherwise. Since we don't add them
anymore from the runtime, the DevTools adds them instead.
Stacked on #29044.
To work with `console.createTask(...).run(...)` we need to be able to
run a function in the scope of the task.
The main concern with this, other than general performance, is that it
might add more stack frames on very deep stacks that hit the stack
limit. Such as with the commit phase where we recursively go down the
tree. These callbacks aren't really necessary in the recursive part but
only in the shallow invocation of the commit phase for each tag. So we
could refactor the commit phase so that only the shallow part at each
level is covered this way.
This one should be fully behind the `enableOwnerStacks` flag.
Instead of printing the parent Component stack all the way to the root,
this now prints the owner stack of every JSX callsite. It also includes
intermediate callsites between the Component and the JSX call so it has
potentially more frames. Mainly it provides the line number of the JSX
callsite. In terms of the number of components is a subset of the parent
component stack so it's less information in that regard. This is usually
better since it's more focused on components that might affect the
output but if it's contextual based on rendering it's still good to have
parent stack. Therefore, I still use the parent stack when printing DOM
nesting warnings but I plan on switching that format to a diff view
format instead (Next.js already reformats the parent stack like this).
__Follow ups__
- Server Components show up in the owner stack for client logs but logs
done by Server Components don't yet get their owner stack printed as
they're replayed. They're also not yet printed in the server logs of the
RSC server.
- Server Component stack frames are formatted as the server and added to
the end but this might be a different format than the browser. E.g. if
server is running V8 and browser is running JSC or vice versa. Ideally
we can reformat them in terms of the client formatting.
- This doesn't yet update Fizz or DevTools. Those will be follow ups.
Fizz still prints parent stacks in the server side logs. The stacks
added to user space `console.error` calls by DevTools still get the
parent stacks instead.
- It also doesn't yet expose these to user space so there's no way to
get them inside `onCaughtError` for example or inside a custom
`console.error` override.
- In another follow up I'll use `console.createTask` instead and
completely remove these stacks if it's available.
Updates Environment#getGlobalDeclaration() to only resolve "globals" if they are a true global or an import from react/react-dom. We still keep the logic to resolve hook-like names as custom hooks. Notably, this means that a local `Array` reference won't get confused with our Array global declaration, a local `useState` (or import from something other than React) won't get confused as `React.useState()`, etc.
I tried to write a proper fixture test to test that we react to changes to a custom setState setter function, but I think there may be an issue with snap and how it handles re-renders from effects. I think the tests are good here but open to feedback if we want to go down the rabbit hole of figuring out a proper snap test for this.
ghstack-source-id: 5e9a8f6e0d23659c72a9d041e8d394b83d6e526d
Pull Request resolved: https://github.com/facebook/react/pull/29190
No-op refactor to make Environment#getGlobalDeclaration() take a NonLocalBinding instead of just a name. The idea is that in subsequent PRs we can use information about the binding to resolve a type more accurately. For example, we can resolve `Array` differently if its an import or local and not the global Array. Similar for resolving local `useState` differently than the one from React.
ghstack-source-id: c8063e6fb8acdd347a56477d6b06238dd54979b1
Pull Request resolved: https://github.com/facebook/react/pull/29189
We currently use `LoadGlobal` and `StoreGlobal` to represent any read (or write) of a variable defined outside the component or hook that is being compiled. This is mostly fine, but for a lot of things we want to do going forward (resolving types across modules, for example) it helps to understand the actual source of a variable.
This PR is an incremental step in that direction. We continue to use LoadGlobal/StoreGlobal, but LoadGlobal now has a `binding:NonLocalBinding` instead of just the name of the global. The NonLocalBinding type tells us whether it was an import (and which kind, the source module name etc), a module-local binding, or a true global. By keeping the LoadGlobal/StoreGlobal instructions, most code that deals with "anything not declared locally" doesn't have to care about the difference. However, code that _does_ want to know the source of the value can figure it out.
ghstack-source-id: e701d4ebc0fb5681a0197198ac2c2a03b3e8aae9
Pull Request resolved: https://github.com/facebook/react/pull/29188
Note: Despite the similar-sounding description, this fix is unrelated to
the issue where updates that occur after an `await` in an async action
must also be wrapped in their own `startTransition`, due to the absence
of an AsyncContext mechanism in browsers today.
---
Discovered a flaw in the current implementation of the isomorphic
startTransition implementation (React.startTransition), related to async
actions. It only creates an async scope if something calls setState
within the synchronous part of the action (i.e. before the first
`await`). I had thought this was fine because if there's no update
during this part, then there's nothing that needs to be entangled. I
didn't think this through, though — if there are multiple async updates
interleaved throughout the rest of the action, we need the async scope
to have already been created so that _those_ are batched together.
An even easier way to observe this is to schedule an optimistic update
after an `await` — the optimistic update should not be reverted until
the async action is complete.
To implement, during the reconciler's module initialization, we compose
its startTransition implementation with any previous reconciler's
startTransition that was already initialized. Then, the isomorphic
startTransition is the composition of every
reconciler's startTransition.
```js
function startTransition(fn) {
return startTransitionDOM(() => {
return startTransitionART(() => {
return startTransitionThreeFiber(() => {
// and so on...
return fn();
});
});
});
}
```
This is basically how flushSync is implemented, too.
Previously Suspensey recursion would only trigger if the
ShouldSuspendCommit flag was true. However there is a dependence on the
Visibility flag embedded in this logic because these flags share a bit.
To make it clear that the semantics of Suspensey resources require
considering both flags I've added it to the condition even though this
extra or-ing is a noop when the bit is shared
This is necessary to simplify the component stack handling to make way
for owner stacks. It also solves some hacks that we used to have but
don't quite make sense. It also solves the problem where things like key
warnings get silenced in RSC because they get deduped. It also surfaces
areas where we were missing key warnings to begin with.
Almost every type of warning is issued from the renderer. React Elements
are really not anything special themselves. They're just lazily invoked
functions and its really the renderer that determines there semantics.
We have three types of warnings that previously fired in
JSX/createElement:
- Fragment props validation.
- Type validation.
- Key warning.
It's nice to be able to do some validation in the JSX/createElement
because it has a more specific stack frame at the callsite. However,
that's the case for every type of component and validation. That's the
whole point of enableOwnerStacks. It's also not sufficient to do it in
JSX/createElement so we also have validation in the renderers too. So
this validation is really just an eager validation but also happens
again later.
The problem with these is that we don't really know what types are valid
until we get to the renderer. Additionally, by placing it in the
isomorphic code it becomes harder to do deduping of warnings in a way
that makes sense for that renderer. It also means we can't reuse logic
for managing stacks etc.
Fragment props validation really should just be part of the renderer
like any other component type. This also matters once we add Fragment
refs and other fragment features. So I moved this into Fiber. However,
since some Fragments don't have Fibers, I do the validation in
ChildFiber instead of beginWork where it would normally happen.
For `type` validation we already do validation when rendering. By
leaving it to the renderer we don't have to hard code an extra list.
This list also varies by context. E.g. class components aren't allowed
in RSC but client references are but we don't have an isomorphic way to
identify client references because they're defined by the host config so
the current logic is flawed anyway. I kept the early validation for now
without the `enableOwnerStacks` since it does provide a nicer stack
frame but with that flag on it'll be handled with nice stacks anyway. I
normalized some of the errors to ensure tests pass.
For `key` validation it's the same principle. The mechanism for the
heuristic is still the same - if it passes statically through a parent
JSX/createElement call then it's considered validated. We already did
print the error later from the renderer so this also disables the early
log in the `enableOwnerStacks` flag.
I also added logging to Fizz so that key warnings can print in SSR logs.
Flight is a bit more complex. For elements that end up on the client we
just pass the `validated` flag along to the client and let the client
renderer print the error once rendered. For server components we log the
error from Flight with the server component as the owner on the stack
which will allow us to print the right stack for context. The factoring
of this is a little tricky because we only want to warn if it's in an
array parent but we want to log the error later to get the right debug
info.
Fiber/Fizz has a similar factoring problem that causes us to create a
fake Fiber for the owner which means the logs won't be associated with
the right place in DevTools.
Repro of a case where we should ideally merge consecutive scopes, but where intermediate temporaries prevent the scopes from merging.
We'd need to reorder instructions in order to merge these.
ghstack-source-id: 4f05672604eeb547fc6c26ef99db6572843ac646
Pull Request resolved: https://github.com/facebook/react/pull/29197
In MergeReactiveScopesThatInvalidateTogether when deciding which scopes were eligible for mergin at all, we looked specifically at the instructions whose lvalue produces the declaration. So if a scope declaration was `t0`, we'd love for the instruction where `t0` was the lvalue and look at the instruction type to decide if it is eligible for merging.
Here, we use the inferred type instead (now that the inferred types support the same set of types of instructions we looked at before). This allows us to find more cases where scopes can be merged.
ghstack-source-id: 0e3e05f24ea0ac6e3c43046bc3e114f906747a04
Pull Request resolved: https://github.com/facebook/react/pull/29157
Improves merging of consecutive scopes so that we now merge two scopes if the dependencies of the second scope are a subset of the previous scope's output *and* that dependency has a type that will always produce a new value (array, object, jsx, function) if it is re-evaluated.
To make this easier, we extend the set of builtin types to include ones for function expressions and JSX and to infer these types in InferTypes. This allows using the already inferred types in MergeReactiveScopesThatInvalidateTogether.
ghstack-source-id: e9119fc4e02b3665848113d71fdff0c5bac3348a
Pull Request resolved: https://github.com/facebook/react/pull/29156
React Compiler attempts to merge consecutive reactive scopes in order to reduce overhead. The basic idea is that if two consecutive scopes would always invalidate together then we should merge them. It gets more complicated, though, because values produced by the earlier scope may not always invalidate when their inputs do. For example, a scope that produces `fn(x)` may not invalidate on all changes to `x` if the function is `Math.max(x, 10)` (changing x from 8 to 9 won't change the output).
Previously we were conservative and only merged if either:
* the two scopes had the same dependencies
* the second scope's deps exactly matched the previous scope's outputs.
You can see this in the new fixture, where the second `<button>` gets its own scope, which happens because the preceding scope has an extra output that isn't a dep of the `<button>`'s scope.
ghstack-source-id: d869c8d4df5aa4105bbdae01b5dd7f106145b351
Pull Request resolved: https://github.com/facebook/react/pull/29155
This lets us expose the component stack to the error reporting that
happens here as `console.error` patching. Now if you just call
`console.error` in the error handlers it'll get the component stack
added to the end by React DevTools.
However, unfortunately this happens a little too late so the Fiber will
be disconnected with its `.return` pointer set to null already. So it'll
be too late to extract a parent component stack from but you can at
least get the stack from source to error boundary. To work around this I
manually add the parent component stack in our default handlers when
owner stacks are off. We could potentially fix this but you can also
just include it yourself if you're calling `console.error` and it's not
a problem for owner stacks.
This is not a problem for owner stacks because we'll still have those
and so for those just calling `console.error` just works. However, the
main feature is that by letting React add them, we can switch to using
native error stacks when available.
We previously had two slightly different concepts for "current fiber".
There's the "owner" which is set inside of class components in prod if
string refs are enabled, and sometimes inside function components in DEV
but not other contexts.
Then we have the "current fiber" which is only set in DEV for various
warnings but is enabled in a bunch of contexts.
This unifies them into a single "current fiber".
The concept of string refs shouldn't really exist so this should really
be a DEV only concept. In the meantime, this sets the current fiber
inside class render only in prod, however, in DEV it's now enabled in
more contexts which can affect the string refs. That was already the
case that a string ref in a Function component was only connecting to
the owner in prod. Any string ref associated with any non-class won't
work regardless so that's not an issue. The practical change here is
that an element with a string ref created inside a life-cycle associated
with a class will work in DEV but not in prod. Since we need the current
fiber to be available in more contexts in DEV for the debugging
purposes. That wouldn't affect any old code since it would have a broken
ref anyway. New code shouldn't use string refs anyway.
The other implication is that "owner" doesn't necessarily mean
"rendering" since we need the "owner" to track other debug information
like stacks - in other contexts like useEffect, life cycles, etc.
Internally we have a separate `isRendering` flag that actually means
we're rendering but even that is a very overloaded concept. So anything
that uses "owner" to imply rendering might be wrong with this change.
This is a first step to a larger refactor for tracking current rendering
information.
---------
Co-authored-by: Sebastian Silbermann <silbermann.sebastian@gmail.com>
## Summary
We ran React compiler against part of our codebase and collected
compiler errors. One of the more common non-actionable errors is caused
by usage of the `!` TypeScript non-null assertion operation:
```
(BuildHIR::lowerExpression) Handle TSNonNullExpression expressions
```
It seems like React Compiler _should_ be able to support this by just
ignoring the syntax and using the underlying expression. I'm sure a lot
of our non-null assertion usage should not exist and I understand if
React Compiler does not want to support this syntax. It wasn't obvious
to me if this omission was intentional or if there are future plans to
use `TSNonNullExpression` as part of the compiler's analysis. If there
are no future plans it seems like just ignoring it should be fine.
## How did you test this change?
```sh
❯ yarn snap --filter
yarn run v1.17.3
$ yarn workspace babel-plugin-react-compiler run snap --filter
$ node ../snap/dist/main.js --filter
PASS non-null-assertion
1 Tests, 1 Passed, 0 Failed
```
In #29201 a fix was made to ensure we don't "forget" about some
listeners when handling cyclic chunks.
In #29204 another fix was made for a special case when the chunk already
has listeners before it first resolves.
This implements the followup fix for Flight Reply which was originally
missed in #29204
Co-authored-by: Janka Uryga <lolzatu2@gmail.com>
Updates Suspensey instances and resources to preload even during urgent
updates and to potentially suspend.
The current implementation is unchanged for transitions but for sync
updates if there is a suspense boundary above the resource/instance it
will be rendered in fallback mode instead.
Note: This behavior is not what we want for images once we make them
suspense enabled. We will need to have forked behavior here to
distinguish between stylesheets which should never commit when not
loaded and images which should commit after a small delay
When using the playground you typically want to see what it outputs, so
let's make the JS tab expanded by default.
ghstack-source-id: 721bc4c381c50db008058b31e1f976e92eab8548
Pull Request resolved: https://github.com/facebook/react/pull/29203
Follow up to https://github.com/facebook/react/pull/29201. If a chunk
had listeners attached already (e.g. because `.then` was called on the
chunk returned from `createFromReadableStream`),
`wakeChunkIfInitialized` would overwrite any listeners added during
chunk initialization. This caused cyclic [path
references](https://github.com/facebook/react/pull/28996) within that
chunk to never resolve. Fixed by merging the two arrays of listeners.
Explicitly waits for the build to finish since the playground requires
them to run
ghstack-source-id: 0bd7d5272d7fa09dc3a140b82a962dc4a3ae585b
Pull Request resolved: https://github.com/facebook/react/pull/29180
Fixes#29200
The cyclic state might have added listeners that will still need to be
invoked. This happens if we have a cyclic reference AND end up blocked.
We have already cleared these before entering the parsing when we enter
the CYCLIC state so we they already have the right type. If listeners
are added during this phase they should carry over to the blocked state.
---------
Co-authored-by: Hendrik Liebau <mail@hendrik-liebau.de>
## Summary
```js
assertConsoleErrorDev([
['Hello', {withoutStack: true}]
])
```
now errors with a helpful diff message if the message mismatched. See
first commit for previous behavior.
## How did you test this change?
- `yarn test --watch
packages/internal-test-utils/__tests__/ReactInternalTestUtils-test.js`
## Summary
Enables the `disableStringRefs` and `enableRefAsProp` feature flags for
React Native (Meta).
## How did you test this change?
```
$ yarn test
$ yarn flow fabric
```
## Summary
Closes#29130
## How did you test this change?
Run the healthcheck in the compiler playground and the nodejs.org repo
for the next config with a `.mjs` extension. Sanity with Vite React
template.
Signed-off-by: abizek <abishekilango@protonmail.com>
<!--
Thanks for submitting a pull request!
We appreciate you spending the time to work on these changes. Please
provide enough information so that others can review your pull request.
The three fields below are mandatory.
Before submitting a pull request, please make sure the following is
done:
1. Fork [the repository](https://github.com/facebook/react) and create
your branch from `main`.
2. Run `yarn` in the repository root.
3. If you've fixed a bug or added code that should be tested, add tests!
4. Ensure the test suite passes (`yarn test`). Tip: `yarn test --watch
TestName` is helpful in development.
5. Run `yarn test --prod` to test in the production environment. It
supports the same options as `yarn test`.
6. If you need a debugger, run `yarn test --debug --watch TestName`,
open `chrome://inspect`, and press "Inspect".
7. Format your code with
[prettier](https://github.com/prettier/prettier) (`yarn prettier`).
8. Make sure your code lints (`yarn lint`). Tip: `yarn linc` to only
check changed files.
9. Run the [Flow](https://flowtype.org/) type checks (`yarn flow`).
10. If you haven't already, complete the CLA.
Learn more about contributing:
https://reactjs.org/docs/how-to-contribute.html
-->
## Summary
<!--
Explain the **motivation** for making this change. What existing problem
does the pull request solve?
-->
In the playground, it's hard to see at a glance what compiler passes are
involved in introducing changes.
This PR bolds every pass that introduces a change.
## How did you test this change?
<!--
Demonstrate the code is solid. Example: The exact commands you ran and
their output, screenshots / videos if the pull request changes the user
interface.
How exactly did you verify that your PR solves the issue you wanted to
solve?
If you leave this empty, your PR will very likely be closed.
-->
Before:
<img width="1728" alt="image"
src="https://github.com/facebook/react/assets/5144292/803ca786-0726-4456-b0db-520dc90a6771">
After:
<img width="1728" alt="image"
src="https://github.com/facebook/react/assets/5144292/38885644-00e9-4065-9c44-db533000d13a">
## Summary
- While rolling out RDT 5.2.0 on Fusebox, we've discovered that context
menus don't work well with this environment. The reason for it is the
context menu state implementation - in a global context we define a map
of registered context menus, basically what is shown at the moment (see
deleted Contexts.js file). These maps are not invalidated on each
re-initialization of DevTools frontend, since the bundle
(react-devtools-fusebox module) is not reloaded, and this results into
RDT throwing an error that some context menu was already registered.
- We should not keep such data in a global state, since there is no
guarantee that this will be invalidated with each re-initialization of
DevTools (like with browser extension, for example).
- The new implementation is based on a `ContextMenuContainer` component,
which will add all required `contextmenu` event listeners to the
anchor-element. This component will also receive a list of `items` that
will be displayed in the shown context menu.
- The `ContextMenuContainer` component is also using
`useImperativeHandle` hook to extend the instance of the component, so
context menus can be managed imperatively via `ref`:
`contextMenu.current?.hide()`, for example.
- **Changed**: The option for copying value to clipboard is now hidden
for functions. The reasons for it are:
- It is broken in the current implementation, because we call
`JSON.stringify` on the value, see
`packages/react-devtools-shared/src/backend/utils.js`.
- I don't see any reasonable value in doing this for the user, since `Go
to definition` option is available and you can inspect the real code and
then copy it.
- We already filter out fields from objects, if their value is a
function, because the whole object is passed to `JSON.stringify`.
## How did you test this change?
### Works with element props and hooks:
- All context menu items work reliably for props items
- All context menu items work reliably or hooks items
https://github.com/facebook/react/assets/28902667/5e2d58b0-92fa-4624-ad1e-2bbd7f12678f
### Works with timeline profiler:
- All context menu items work reliably: copying, zooming, ...
- Context menu automatically closes on the scroll event
https://github.com/facebook/react/assets/28902667/de744cd0-372a-402a-9fa0-743857048d24
### Works with Fusebox:
- Produces no errors
- Copy to clipboard context menu item works reliably
https://github.com/facebook/react/assets/28902667/0288f5bf-0d44-435c-8842-6b57bc8a7a24
Bumps [rustix](https://github.com/bytecodealliance/rustix) from 0.37.22
to 0.37.27.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="b38dc51262"><code>b38dc51</code></a>
chore: Release rustix version 0.37.27</li>
<li><a
href="a2d9c8ee1a"><code>a2d9c8e</code></a>
Fix p{read,write}v{,v2}'s encoding of the offset argument on Linux. (<a
href="https://redirect.github.com/bytecodealliance/rustix/issues/896">#896</a>)
(#...</li>
<li><a
href="dce2777622"><code>dce2777</code></a>
chore: Release rustix version 0.37.26</li>
<li><a
href="06dbe83c60"><code>06dbe83</code></a>
Fix <code>sendmsg_unix</code>'s address encoding. (<a
href="https://redirect.github.com/bytecodealliance/rustix/issues/885">#885</a>)
(<a
href="https://redirect.github.com/bytecodealliance/rustix/issues/886">#886</a>)</li>
<li><a
href="00b84d6aac"><code>00b84d6</code></a>
chore: Release rustix version 0.37.25</li>
<li><a
href="cad15a7076"><code>cad15a7</code></a>
Fixes for <code>Dir</code> on macOS, FreeBSD, and WASI.</li>
<li><a
href="df3c3a192c"><code>df3c3a1</code></a>
Merge pull request from GHSA-c827-hfw6-qwvm</li>
<li><a
href="b78aeff1a2"><code>b78aeff</code></a>
chore: Release rustix version 0.37.24</li>
<li><a
href="c0c3f01d7c"><code>c0c3f01</code></a>
Add GNU/Hurd support (<a
href="https://redirect.github.com/bytecodealliance/rustix/issues/852">#852</a>)</li>
<li><a
href="f416b6b27b"><code>f416b6b</code></a>
Fix the <code>test_ttyname_ok</code> test when /dev/stdin is
inaccessable. (<a
href="https://redirect.github.com/bytecodealliance/rustix/issues/821">#821</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/bytecodealliance/rustix/compare/v0.37.22...v0.37.27">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/facebook/react/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [postcss](https://github.com/postcss/postcss) from 8.4.24 to
8.4.31.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/postcss/postcss/releases">postcss's
releases</a>.</em></p>
<blockquote>
<h2>8.4.31</h2>
<ul>
<li>Fixed <code>\r</code> parsing to fix CVE-2023-44270.</li>
</ul>
<h2>8.4.30</h2>
<ul>
<li>Improved source map performance (by <a
href="https://github.com/romainmenke"><code>@romainmenke</code></a>).</li>
</ul>
<h2>8.4.29</h2>
<ul>
<li>Fixed <code>Node#source.offset</code> (by <a
href="https://github.com/idoros"><code>@idoros</code></a>).</li>
<li>Fixed docs (by <a
href="https://github.com/coliff"><code>@coliff</code></a>).</li>
</ul>
<h2>8.4.28</h2>
<ul>
<li>Fixed <code>Root.source.end</code> for better source map (by <a
href="https://github.com/romainmenke"><code>@romainmenke</code></a>).</li>
<li>Fixed <code>Result.root</code> types when <code>process()</code> has
no parser.</li>
</ul>
<h2>8.4.27</h2>
<ul>
<li>Fixed <code>Container</code> clone methods types.</li>
</ul>
<h2>8.4.26</h2>
<ul>
<li>Fixed clone methods types.</li>
</ul>
<h2>8.4.25</h2>
<ul>
<li>Improve stringify performance (by <a
href="https://github.com/romainmenke"><code>@romainmenke</code></a>).</li>
<li>Fixed docs (by <a
href="https://github.com/vikaskaliramna07"><code>@vikaskaliramna07</code></a>).</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/postcss/postcss/blob/main/CHANGELOG.md">postcss's
changelog</a>.</em></p>
<blockquote>
<h2>8.4.31</h2>
<ul>
<li>Fixed <code>\r</code> parsing to fix CVE-2023-44270.</li>
</ul>
<h2>8.4.30</h2>
<ul>
<li>Improved source map performance (by Romain Menke).</li>
</ul>
<h2>8.4.29</h2>
<ul>
<li>Fixed <code>Node#source.offset</code> (by Ido Rosenthal).</li>
<li>Fixed docs (by Christian Oliff).</li>
</ul>
<h2>8.4.28</h2>
<ul>
<li>Fixed <code>Root.source.end</code> for better source map (by Romain
Menke).</li>
<li>Fixed <code>Result.root</code> types when <code>process()</code> has
no parser.</li>
</ul>
<h2>8.4.27</h2>
<ul>
<li>Fixed <code>Container</code> clone methods types.</li>
</ul>
<h2>8.4.26</h2>
<ul>
<li>Fixed clone methods types.</li>
</ul>
<h2>8.4.25</h2>
<ul>
<li>Improve stringify performance (by Romain Menke).</li>
<li>Fixed docs (by <a
href="https://github.com/vikaskaliramna07"><code>@vikaskaliramna07</code></a>).</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="90208de880"><code>90208de</code></a>
Release 8.4.31 version</li>
<li><a
href="58cc860b4c"><code>58cc860</code></a>
Fix carrier return parsing</li>
<li><a
href="4fff8e4cdc"><code>4fff8e4</code></a>
Improve pnpm test output</li>
<li><a
href="cd43ed1232"><code>cd43ed1</code></a>
Update dependencies</li>
<li><a
href="caa916bdcb"><code>caa916b</code></a>
Update dependencies</li>
<li><a
href="8972f76923"><code>8972f76</code></a>
Typo</li>
<li><a
href="11a5286f78"><code>11a5286</code></a>
Typo</li>
<li><a
href="45c5501777"><code>45c5501</code></a>
Release 8.4.30 version</li>
<li><a
href="bc3c341f58"><code>bc3c341</code></a>
Update linter</li>
<li><a
href="b2be58a2eb"><code>b2be58a</code></a>
Merge pull request <a
href="https://redirect.github.com/postcss/postcss/issues/1881">#1881</a>
from romainmenke/improve-sourcemap-performance--phil...</li>
<li>Additional commits viewable in <a
href="https://github.com/postcss/postcss/compare/8.4.24...8.4.31">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/facebook/react/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
By default, React Compiler will skip compilation if it cannot preserve existing memoization. Ie, if the code has an existing `useMemo()` or `useCallback()` and the compiler cannot determine that it is safe to keep that memoization — or do even better — then we'll leave the code alone. The actual compilation doesn't use any hints from existing memo calls, this is purely to check and avoid regressing any specific memoization that developers may have already applied.
However, we were accidentally reporting some false-positive _validation_ errors due to the StartMemoize and FinishMemoize instructions that we emit to track where the memoization was in the source code. This is now fixed.
Fixes#29131Fixes#29132
ghstack-source-id: 9f6b8dbc5074ccc96e6073cf11c4920b5375faf6
Pull Request resolved: https://github.com/facebook/react/pull/29154
Improves ValidateNoRefAccessInRender (still disabled by default) to properly ignore ref access within effects. This includes allowing ref access within functions that are only transitively called from an effect.
While I was here I also added some extra test fixtures for allowing global mutation in effects.
ghstack-source-id: fb6352a1788b7bdbebb40d5b844b711ef87d6771
Pull Request resolved: https://github.com/facebook/react/pull/29151
As a fellow beginner to React, I didn't even know React runs on top of
Node when I started.
So, some beginners might get confused about what is Node and how to find
details about it or how to download it.
So, I thought to add a hyperlink to Node replacing the word Node in the
README.md file. I think this might be a valuable contribution.
- Januda
## Summary
The "Good First Issues" header in the README was missing a hyperlink
where the other similar headlines had one.
<!--
Explain the **motivation** for making this change. What existing problem
does the pull request solve?
-->
## How did you test this change?
N/A
<!--
Demonstrate the code is solid. Example: The exact commands you ran and
their output, screenshots / videos if the pull request changes the user
interface.
How exactly did you verify that your PR solves the issue you wanted to
solve?
If you leave this empty, your PR will very likely be closed.
-->
Makes running the script a little more ergonomic by prompting for OTP
upfront.
ghstack-source-id: e9967bfde1ab01ff9417a848154743ae1926318d
Pull Request resolved: https://github.com/facebook/react/pull/29149
Babel doesn't seem to properly preserve escaping of HTML entities when emitting JSX text children, so this commit works around the issue by emitting a JsxExpressionContainer for JSX children that contain ">", "<", or "&" characters.
Closes#29100
ghstack-source-id: 2d0622397cc067c6336f3635073e07daef854084
Pull Request resolved: https://github.com/facebook/react/pull/29143
## Summary
Every tab wraps the text around but there is no way to resize it. It was
also hard to use the source map tab. It doesn't occupy the full height
nor is the tab resizable. So I made all the tabs resizable.
> Also,
> * make the source map tab occupy full height
> * make it a teeny tiny bit easier to work with the compiler playground
(especially source map)
## How did you test this change?
https://github.com/facebook/react/assets/91976421/cdec30e8-cadb-4958-8786-31c54ea83bd6
Signed-off-by: abizek <abishekilango@protonmail.com>
Adds a GitHub issue template form so we can automatically categorize
issues and get more information upfront. I mostly referenced the
DevTools bug report template and made some tweaks.
ghstack-source-id: 5bfc728a625f367932fc21263e82681079d3ac65
Pull Request resolved: https://github.com/facebook/react/pull/29140
Workaround for a bug in older versions of Babel, where strings with unicode are incorrectly escaped when emitted as JSX attributes, causing double-escaping by later processing.
Closes#29120Closes#29124
ghstack-source-id: 065440d4fb97e164beb8a8f15f252f372a59c5a0
Pull Request resolved: https://github.com/facebook/react/pull/29141
Now that the compiler is public, the `*` version was grabbing the latest
version of the compiler off of npm and was resolving to my very first
push to npm (an empty package containing only a single package.json).
This was breaking the playground as it would attempt to load the
compiler but then crash the babel pipeline due to the node module not
being found.
ghstack-source-id: 695fd9caac
Pull Request resolved: https://github.com/facebook/react/pull/29122
@jbonta nerd-sniped me into making this optimization during conference
prep, posting this as a PR now that keynote is over.
Consider these two cases:
```javascript
export default function MyApp1({ count }) {
const cb = () => count;
return <div onclick={cb}>Hello World</div>;
}
export default function MyApp2({ count }) {
return <div onclick={() => count}>Hello World</div>;
}
```
Previously, the former would create two reactive scopes (one for `cb`,
one for the div) while the latter would only have a single scope for the
`div` and its inline callback. The reason we created separate scopes
before is that there's a `StoreLocal 'cb' = t0` instruction in-between,
and i had conservatively implemented the merging pass to not allow
intervening StoreLocal instructions.
The observation is that intervening StoreLocals are fine _if_ the
assigned variable's last usage is the next scope. We already have a
check that the intervening lvalues are last-used at/before the next
scope, so it's trivial to extend this to support StoreLocal.
Note that we already don't merge scopes if there are intervening
terminals, so we don't have to worry about things like conditional
StoreLocal, conditional access of the resulting value, etc.
/cc @poteto
<!--
Thanks for submitting a pull request!
We appreciate you spending the time to work on these changes. Please
provide enough information so that others can review your pull request.
The three fields below are mandatory.
Before submitting a pull request, please make sure the following is
done:
1. Fork [the repository](https://github.com/facebook/react) and create
your branch from `main`.
2. Run `yarn` in the repository root.
3. If you've fixed a bug or added code that should be tested, add tests!
4. Ensure the test suite passes (`yarn test`). Tip: `yarn test --watch
TestName` is helpful in development.
5. Run `yarn test --prod` to test in the production environment. It
supports the same options as `yarn test`.
6. If you need a debugger, run `yarn test --debug --watch TestName`,
open `chrome://inspect`, and press "Inspect".
7. Format your code with
[prettier](https://github.com/prettier/prettier) (`yarn prettier`).
8. Make sure your code lints (`yarn lint`). Tip: `yarn linc` to only
check changed files.
9. Run the [Flow](https://flowtype.org/) type checks (`yarn flow`).
10. If you haven't already, complete the CLA.
Learn more about contributing:
https://reactjs.org/docs/how-to-contribute.html
-->
## Summary
<!--
Explain the **motivation** for making this change. What existing problem
does the pull request solve?
-->
Seems like the README of the package was outdated.
## How did you test this change?
Tried this configuration in a project of mine.
<!--
Demonstrate the code is solid. Example: The exact commands you ran and
their output, screenshots / videos if the pull request changes the user
interface.
How exactly did you verify that your PR solves the issue you wanted to
solve?
If you leave this empty, your PR will very likely be closed.
-->
<!--
Thanks for submitting a pull request!
We appreciate you spending the time to work on these changes. Please
provide enough information so that others can review your pull request.
The three fields below are mandatory.
Before submitting a pull request, please make sure the following is
done:
1. Fork [the repository](https://github.com/facebook/react) and create
your branch from `main`.
2. Run `yarn` in the repository root.
3. If you've fixed a bug or added code that should be tested, add tests!
4. Ensure the test suite passes (`yarn test`). Tip: `yarn test --watch
TestName` is helpful in development.
5. Run `yarn test --prod` to test in the production environment. It
supports the same options as `yarn test`.
6. If you need a debugger, run `yarn test --debug --watch TestName`,
open `chrome://inspect`, and press "Inspect".
7. Format your code with
[prettier](https://github.com/prettier/prettier) (`yarn prettier`).
8. Make sure your code lints (`yarn lint`). Tip: `yarn linc` to only
check changed files.
9. Run the [Flow](https://flowtype.org/) type checks (`yarn flow`).
10. If you haven't already, complete the CLA.
Learn more about contributing:
https://reactjs.org/docs/how-to-contribute.html
-->
## Summary
Use `filename` instead of `context.filename` in eslint compiler.
The problem is that in `react-native` + `typescript` project the context
may not have `filename`:
<img width="384" alt="image"
src="https://github.com/facebook/react/assets/22820318/e5d184fa-5ac9-4512-96b9-644baa3d5f25">
And eslint will crash with:
```bash
TypeError: Error while loading rule 'react-compiler/react-compiler': Cannot read properties of undefined (reading 'endsWith')
```
But in fact we already derive `filename` variable earlier so we can
simply reuse the variable (I guess).
## How did you test this change?
- add `eslint` plugin to RN project;
- run eslint
Uses https for the npm registry so the publishing script isn't rejected.
Fixes:
```
Beginning October 4, 2021, all connections to the npm registry - including for package installation - must use TLS 1.2 or higher. You are currently using plaintext http to connect. Please visit the GitHub blog for more information: https://github.blog/2021-08-23-npm-registry-deprecating-tls-1-0-tls-1-1/
```
ghstack-source-id: b247d044ea48f3007cf8bd13445fb7ece0f5b02f
Pull Request resolved: https://github.com/facebook/react/pull/29087
<!--
Thanks for submitting a pull request!
We appreciate you spending the time to work on these changes. Please
provide enough information so that others can review your pull request.
The three fields below are mandatory.
Before submitting a pull request, please make sure the following is
done:
1. Fork [the repository](https://github.com/facebook/react) and create
your branch from `main`.
2. Run `yarn` in the repository root.
3. If you've fixed a bug or added code that should be tested, add tests!
4. Ensure the test suite passes (`yarn test`). Tip: `yarn test --watch
TestName` is helpful in development.
5. Run `yarn test --prod` to test in the production environment. It
supports the same options as `yarn test`.
6. If you need a debugger, run `yarn test --debug --watch TestName`,
open `chrome://inspect`, and press "Inspect".
7. Format your code with
[prettier](https://github.com/prettier/prettier) (`yarn prettier`).
8. Make sure your code lints (`yarn lint`). Tip: `yarn linc` to only
check changed files.
9. Run the [Flow](https://flowtype.org/) type checks (`yarn flow`).
10. If you haven't already, complete the CLA.
Learn more about contributing:
https://reactjs.org/docs/how-to-contribute.html
-->
## Summary
This PR fixes a typo in `./compiler/docs/DESIGN_GOALS.md`.
I believe `plugion` should be `plugin`.
## How did you test this change?
Rendered the markdown to html.
This script needs to run from `main` since it commits version bumps for
packages, and those need to point to publicly available hashes. So,
throw an error if we're not already on main.
ghstack-source-id: ce0168e826
Pull Request resolved: https://github.com/facebook/react/pull/29083
- Specify a registry for npm publish because otherwise it tries to use
the yarn registry
- `packages` option actually works
This _should_ work now (note last line of output), will test it once we
land this since i want to publish a new version of the eslint plugin
with some important fixes.
```
npm notice
npm notice 📦 eslint-plugin-react-compiler@0.0.0-experimental-53bb89e-20240515
npm notice === Tarball Contents ===
npm notice 827B README.md
npm notice 2.1MB dist/index.js
npm notice 1.0kB package.json
npm notice === Tarball Details ===
npm notice name: eslint-plugin-react-compiler
npm notice version: 0.0.0-experimental-53bb89e-20240515
npm notice filename: eslint-plugin-react-compiler-0.0.0-experimental-53bb89e-20240515.tgz
npm notice package size: 300.9 kB
npm notice unpacked size: 2.1 MB
npm notice shasum: cb99823f3a483c74f470085cac177bd020f7a85a
npm notice integrity: sha512-L3HV9qja1dnCl[...]IaRSZJ3P/v6yQ==
npm notice total files: 3
npm notice
npm notice Publishing to http://registry.npmjs.org/ with tag latest and default access (dry-run)
```
ghstack-source-id: 63067ef772
Pull Request resolved: https://github.com/facebook/react/pull/29082
Previously we would attempt to parse code in the eslint plugin with the
HermesParser first as it can handle some TS syntax. However, this was
leading to a mis-parse of React hook calls with type params (eg,
`useRef<null>()` as a BinaryExpression rather than a CallExpression with
a type param. This triggered our validation that Hooks should not be
used as normal values.
To fix this, we now try to parse with the babel parser (with TS support)
for filenames that end with ts/tsx, and fallback to HermesParser for
regular JS files.
ghstack-source-id: 5b7231031c
Pull Request resolved: https://github.com/facebook/react/pull/29081
Previously, we only checked for StrictMode by searching for
`<StrictMode>` but we should also check for the namespaced version,
`<React.StrictMode>`.
Fixes https://github.com/facebook/react/issues/29075
Fixes the top-level ESLint and Prettier configs to ignore the compiler.
For now the compiler has its own prettier and linting setup with
different versions/configs.
## Summary
The main field is missing, this fixes it.
Fixes#29068.
## How did you test this change?
Manually patched the package and tried it in my codebase.
This updates the Canary label from "beta" to "rc".
We will publish an actual RC (e.g. 19.0.0-rc.0) too; this only changes
the label in the canary releases.
The [`files` field](https://docs.npmjs.com/cli/v10/commands/npm-publish#files-included-in-package)
controls what files get included in the published package.
This PR specifies the `files` field on our publishable packages to only
include the `dist` directory, since we don't need to ship any types or
sourcemaps with 3 of them.
react-compiler-runtime is a runtime package which has sourcemaps, so we
also include the `src` directory in the published package.
Also fixes an invalid version range for the react peer dependency in
react-compiler-runtime, tested that it works via https://semver.npmjs.com/
ghstack-source-id: 12b36c203fc9fd8d72a1995fb3fba2312de4aa51
Pull Request resolved: https://github.com/facebook/react-forget/pull/2965
## Summary
Enables the `enableUnifiedSyncLane` feature flag for React Native
(Meta).
## How did you test this change?
```
$ yarn test
$ yarn flow fabric
```
`forget_napi` doesn't exist and given we're not currently working on the Rust compiler, I'm not sure we need to keep this around until we know that we do want to invest into this area again.
Don't really have time to implement the react-compiler/healthcheck
version of this script, so for now i propose we just publish this as
react-compiler-healthcheck
the command for running this would be
```
$ npx react-compiler-healthcheck --src 'whatever/**/*.*'
```
ghstack-source-id: e2c443a912
Pull Request resolved: https://github.com/facebook/react-forget/pull/2956
runReactBabelPluginReactCompiler brings in fbt which is unnecessary for
OSS so I removed it.
Also makes it so healthckeck is installed as an executable
ghstack-source-id: ec6c76f8be
Pull Request resolved: https://github.com/facebook/react-forget/pull/2955
We found this issue through enabling the compiler on the React Conf app.
`babel-preset-expo` automatically adds the `react-native-animated`
plugin to apps that use the preset. This means that Expo apps sometimes
omit the react-native-animated plugin from their config, which was
failing our existing check. This PR copies the same detection that Expo
does for adding reanimated as a fallback
ghstack-source-id: 46f7aec0bc
Pull Request resolved: https://github.com/facebook/react-forget/pull/2953
This errors on the client normally but in the case the `type` is a
function - i.e. a Server Component - it wouldn't be transferred to error
on the client so you end up with a worse error message. So this just
implements the same check as ChildFiber.
## Summary
The experiment has shown no significant performance changes. This PR
removes it.
## How did you test this change?
```
yarn flow native
yarn lint
```
Stacked on #28997.
We can use the technique of referencing an object by its row + property
name path for temporary references - like we do for deduping. That way
we don't need to generate an ID for temporary references. Instead, they
can just be an opaque marker in the slot and it has the implicit ID of
the row + path.
Then we can stash all objects, even the ones that are actually available
to read on the server, as temporary references. Without adding anything
to the payload since the IDs are implicit. If the same object is
returned to the client, it can be referenced by reference instead of
serializing it back to the client. This also helps preserve object
identity.
We assume that the objects are immutable when they pass the boundary.
I'm not sure if this is worth it but with this mechanism, if you return
the `FormData` payload from a `useActionState` it doesn't have to be
serialized on the way back to the client. This is a common pattern for
having access to the last submission as "default value" to the form
fields. However you can still control it by replacing it with another
object if you want. In MPA mode, the temporary references are not
configured and so it needs to be serialized in that case. That's
required anyway for hydration purposes.
I'm not sure if people will actually use this in practice though or if
FormData will always be destructured into some other object like with a
library that turns it into typed data, and back. If so, the object
identity is lost.
Uses the same technique as in #28996 to encode references to already
emitted objects. This now means that Reply can support cyclic objects
too for parity.
Instead of forcing an object to be outlined to be able to refer to it
later we can refer to it by the property path inside another parent
object.
E.g. this encodes such a reference as `'$123:props:children:foo:bar'`.
That way we don't have to preemptively outline object and we can dedupe
after the first time we've found it.
There's no cost on the client if it's not used because we're not storing
any additional information preemptively.
This works mainly because we only have simple JSON objects from the root
reference. Complex objects like Map, FormData etc. are stored as their
entries array in the look up and not the complex object. Other complex
objects like TypedArrays or imports don't have deeply nested objects in
them that can be referenced.
This solves the problem that we only dedupe after the third instance.
This dedupes at the second instance. It also solves the problem where
all nested objects inside deduped instances also are outlined.
The property paths can get pretty large. This is why a test on payload
size increased. We could potentially outline the reference itself at the
first dupe. That way we get a shorter ID to refer to in the third
instance.
Adds supports for hot module reloading (HMR) by resetting the cache if a hash of the source file changes. This is enabled via a compiler flag, but also enabled automatically via the babel plugin when NODE_ENV=development.
ghstack-source-id: 5cd1ad5c89
Pull Request resolved: https://github.com/facebook/react-forget/pull/2951
Before this change, `useFormStatus` is only activated if a form is
submitted by an action function (either `<form action={actionFn}>` or
`<button formAction={actionFn}>`).
After this change, `useFormStatus` will also be activated if you call
`startTransition(actionFn)` inside a submit event handler that is
`preventDefault`-ed.
This is the last missing piece for implementing a custom `action` prop
that is progressively enhanced using `onSubmit` while maintaining the
same behavior as built-in form actions.
Here's the basic recipe for implementing a progressively-enhanced form
action. This would typically be implemented in your UI component
library, not regular application code:
```js
import {requestFormReset} from 'react-dom';
// To implement progressive enhancement, pass both a form action *and* a
// submit event handler. The action is used for submissions that happen
// before hydration, and the submit handler is used for submissions that
// happen after.
<form
action={action}
onSubmit={(event) => {
// After hydration, we upgrade the form with additional client-
// only behavior.
event.preventDefault();
// Manually dispatch the action.
startTransition(async () => {
// (Optional) Reset any uncontrolled inputs once the action is
// complete, like built-in form actions do.
requestFormReset(event.target);
// ...Do extra action-y stuff in here, like setting a custom
// optimistic state...
// Call the user-provided action
const formData = new FormData(event.target);
await action(formData);
});
}}
/>
```
This is the first step to experimenting with a new type of stack traces
behind the `enableOwnerStacks` flag - in DEV only.
The idea is to generate stacks that are more like if the JSX was a
direct call even though it's actually a lazy call. Not only can you see
which exact JSX call line number generated the erroring component but if
that's inside an abstraction function, which function called that
function and if it's a component, which component generated that
component. For this to make sense it really need to be the "owner" stack
rather than the parent stack like we do for other component stacks. On
one hand it has more precise information but on the other hand it also
loses context. For most types of problems the owner stack is the most
useful though since it tells you which component rendered this
component.
The problem with the platform in its current state is that there's two
ways to deal with stacks:
1) `new Error().stack`
2) `console.createTask()`
The nice thing about `new Error().stack` is that we can extract the
frames and piece them together in whatever way we want. That is great
for constructing custom UIs like error dialogs. Unfortunately, we can't
take custom stacks and set them in the native UIs like Chrome DevTools.
The nice thing about `console.createTask()` is that the resulting stacks
are natively integrated into the Chrome DevTools in the console and the
breakpoint debugger. They also automatically follow source mapping and
ignoreLists. The downside is that there's no way to extract the async
stack outside the native UI itself so this information cannot be used
for custom UIs like errors dialogs. It also means we can't collect this
on the server and then pass it to the client for server components.
The solution here is that we use both techniques and collect both an
`Error` object and a `Task` object for every JSX call.
The main concern about this approach is the performance so that's the
main thing to test. It's certainly too slow for production but it might
also be too slow even for DEV.
This first PR doesn't actually use the stacks yet. It just collects them
as the first step. The next step is to start utilizing this information
in error printing etc.
For RSC we pass the stack along across over the wire. This can be
concatenated on the client following the owner path to create an owner
stack leading back into the server. We'll later use this information to
restore fake frames on the client for native integration. Since this
information quickly gets pretty heavy if we include all frames, we strip
out the top frame. We also strip out everything below the functions that
call into user space in the Flight runtime. To do this we need to figure
out the frames that represents calling out into user space. The
resulting stack is typically just the one frame inside the owner
component's JSX callsite. I also eagerly strip out things we expect to
be ignoreList:ed anyway - such as `node_modules` and Node.js internals.
## Summary
This brings:
- jest* up from 29.4.2 -> 29.7.0
- jsdom up from 20.0.0 -> 22.1.0
While the latest version of jest-dom-environment still wants
`jsdom@^20.0.0`, it can safely use at least up to `jsdom@22.1.0`. See
https://github.com/jestjs/jest/pull/13825#issuecomment-1564015010 for
details.
Upgrading to latest versions lets us improve some WheelEvent tests and
will make it possible to test a much simpler FormData construction
approach (see #29018)
## How did you test this change?
Ran `yarn test` and `yarn test --prod` successfully
Facebook: merge react index.classic.fb and index.modern.fb
These export the same.
NOTE: The 2 builds are still different based on flags and other forked
files.
## Summary
This PR makes some fixes to the `fastAddProperties` function:
- Use `if (!attributeConfig)` instead of `if (attributeConfig ===
undefined)` to account for `null`.
- If a prop has an Object `attributeConfig` with a `diff` function
defined on it, treat it as an atomic value to keep the semantics of
`diffProperties`.
## How did you test this change?
Build and run RNTester app.
This is the same change as #28780 but for the Flight Reply receiver.
While it's not possible to create an "async module" reference in this
case - resolving a server reference can still be async if loading it
requires loading chunks like in a new server instance.
Since extracting a typed array from a Blob is async, that's also a case
where a dependency can be async.
This follows the same principle as in #28611.
We cannot serialize Blobs of a form data into HTML because you can't
initialize a file input to some value. However the serialization of
state in an Action can contain blobs. In this case we do error but
outside the try/catch that recovers to error to client replaying instead
of MPA mode. This errors earlier to ensure that this works.
Testing this is a bit annoying because JSDOM doesn't have any of the
Blob methods but the Blob needs to be compatible with FormData and the
FormData needs to be compatible with `<form>` nodes in these tests. So I
polyfilled those in JSDOM with some hacks.
A possible future enhancement would be to encode these blobs in a base64
mode instead and have some way to receive them on the server. It's just
a matter of layering this. I think the RSC layer's `FORM_DATA`
implementation can pass some flag to encode as base64 and then have
decodeAction include some way to parse them. That way this case would
work in MPA mode too.
Based on #28893.
For other streams we encode each chunk as a separate form field which is
a bit bloated. Especially for binary chunks since they also have an
indirection. We need some way to encode the chunks as separate anyway.
This way the streaming using busboy actually allows each chunk to stream
in over the network one at a time.
For binary streams the actual chunking is not important. The chunks can
be split and recombined in whatever size chunk makes sense.
Since we buffer the entire content anyway we can combine the chunks to
be consecutive. This PR does that with binary streams and also combine
them into a single Blob. That way there's no extra overhead when passing
through a binary stream.
Ideally, we'd be able to just use the stream from that one Blob but
Node.js doesn't return byob streams from Blob. Additionally, we don't
actually stream the content of Blobs due to the layering with busboy
atm. We could do that for binary streams in particular by replacing the
File layering with a stream and resolving each chunk as it comes in.
That could be a follow up.
If we stop buffering in the future, this set up still allows us to split
them and send other form fields in between while blocked since the
protocol is still the same.
Follow-up to https://github.com/facebook/react/pull/28813.
RDT is using `typeOf` from `react-is` to determine the element display
name, I've forked an implementation of this method, but will be using
legacy element symbol.
## Summary
Exposes the APIs needed by React Native DevTools (Fusebox) to implement
the "view element source" and "view attribute source" features.
## How did you test this change?
1. `yarn build` in `react-devtools-fusebox`
2. Copy artifacts to rn-chrome-devtools-frontend
3. Write some additional glue code to implement
`viewElementSourceFunction` in our CDT fork.
4. Test the feature manually.
https://github.com/facebook/react/assets/2246565/12667018-100a-4b3f-957a-06c07f2af41a
We need this in the root to run the steps. It should merge cleanly with the React repo as there is no file name overlap.
Next step is to update the paths to make it work again.
## Summary
This PR introduces Fabric-only version of
`ReactNativeAttributesPayload`. It is a copy-paste of
`ReactNativeAttributesPayload.js`, and is called
`ReactNativeAttributesPayloadFabric.js`.
The idea behind this change is that certain optimizations in prop
diffing may actually be a regression on the old architecture. For
example, removing custom diffing may result in larger updateProps
payloads. Which is, I guess, fine with JSI, but might be a problem with
the bridge.
## How did you test this change?
There should be no runtime effect of this change.
The eslint rule seems to false positive on this typescript syntax, but
strangely the compiler does not
ghstack-source-id: 19baa24ff7addd83f59e2b03fdb180af169a2794
Pull Request resolved: https://github.com/facebook/react-forget/pull/2913
Make it clearer how to address this error by allowlisting globals that
are known to be safe
ghstack-source-id: e7fa6464ebb561a7a1366ff70430842007c6552e
Pull Request resolved: https://github.com/facebook/react-forget/pull/2909
During the demo I might show an example of fixing a
CannotPreserveMemoization error. But I don't want to make that
reportable by default, so this PR allows configuration like so
```js
module.exports = {
root: true,
plugins: [
'eslint-plugin-react-compiler',
],
rules: {
'react-compiler/react-compiler': [
'error', {
reportableLevels: new Set([
'InvalidJs',
'InvalidReact',
'CannotPreserveMemoization'
])
}
]
}
}
```
ghstack-source-id: 984c6d3cb7e19c8fea2bb88108dd26335c031573
Pull Request resolved: https://github.com/facebook/react-forget/pull/2936
We control what gets reported via another function anyway so it's better
to raise everything at the compiler config level. This lets us configure
what level of diagnostic is reportable later
ghstack-source-id: 996d3cbb8d8f3e1bbe943210b8d633420e0f3f3b
Pull Request resolved: https://github.com/facebook/react-forget/pull/2935
This file is intended to test that the test will skip if it was already compiled.
This updates the import to remove the no longer used name `unstable_useMemoCache`.
- Updated all directly defined dependencies to the latest React 19 Beta
- `package.json`: used `resolutions` to force React 19 for `react-is` transitive dependency
- `package.json`: postinstall script to patch fbt for the React 19 element Symbol
- Match on the message in Snap to exclude a React 19 warning that `act` should be imported from `react` instead (from inside `@testing-library/react`)
- Some updated snapshots, I think due to now recovering behavior of `useMemoCache`, please review.
In a next step, we can do the following. I excluded it since it from here as it made the PR unreviewable on GitHub.
- Snapshots now use `react/compiler-runtime` as in prod, so the different default in Snap is no longer needed.
## Summary
Sets up dynamic feature flags for `disableStringRefs`, `enableFastJSX`,
and `enableRefAsProp` in React Native (at Meta).
## How did you test this change?
```
$ yarn test
$ yarn flow fabric
```
In order to integrate the `react-reconciler` build created in #28880
with third party libraries, we need to have matching
`react-reconciler/constants` to go with it.
Same as #28847 but in the other direction.
Like other promises, this doesn't actually stream in the outgoing
direction. It buffers until the stream is done. This is mainly due to
our protocol remains compatible with Safari's lack of outgoing streams
until recently.
However, the stream chunks are encoded as separate fields and so does
support the busboy streaming on the receiving side.
We currently don't test FormData / File dependent features in CI because
we use an old Node.js version in CI. We should probably upgrade to 18
since that's really the minimum version that supports all the features
out of the box.
JSDOM is not a faithful/compatible implementation of these APIs. The
recommended way to use Flight together with FormData/Blob/File in older
Node.js versions, is to polyfill using the `undici` library.
However, even in these versions the Blob implementation isn't quite
faithful so the Reply client needs a slight tweak for multi-byte typed
arrays.
Stacked on #28798.
Add another AsyncLocalStorage to the FlightServerConfig. This context
tracks data on a per component level. Currently the only thing we track
is the owner in DEV.
AsyncLocalStorage around each component comes with a performance cost so
we only do it DEV. It's not generally a particularly safe operation
because you can't necessarily associate side-effects with a component
based on execution scope. It can be a lazy initializer or cache():ed
code etc. We also don't support string refs anymore for a reason.
However, it's good enough for optional dev only information like the
owner.
Bundle config: inline internal hook wrapper
Instead of reading this wrapper from 2 files for "start" and "end" and
then string modifying the templates, just inline them like the other
wrappers in this file.
## Summary
Enables the inspected element context menu in React Native DevTools
(Fusebox).
## How did you test this change?
1. `yarn build` in `react-devtools-fusebox`
2. Copy artifacts to rn-chrome-devtools-frontend
3. Manually test the context menu
https://github.com/facebook/react/assets/2246565/b35cc20f-8d67-43b0-b863-7731e10fffac
NOTE: The serialised values sometimes expose React internals (e.g. Hook
data structures instead of just the values), but that seems to be a
problem equally on web, so I'm going for native<->web parity here.
## Summary
The `react-devtools-fusebox` private package is used in the React Native
DevTools (Fusebox) frontend by checking build artifacts into RN's
[fork]([`facebookexperimental/rn-chrome-devtools-frontend`](https://github.com/facebookexperimental/rn-chrome-devtools-frontend))
of the Chrome DevTools (CDT) repo - see
https://github.com/facebookexperimental/rn-chrome-devtools-frontend/pull/22.
Currently, the CDT fork also includes a [manually written TypeScript
definition
file](1d5f8d5209/front_end/third_party/react-devtools/package/frontend.d.ts)
which describes `react-devtools-fusebox`'s API. This PR moves that file
into the React repo, next to the implementation of
`react-devtools-fusebox`, so we can update it atomically with changes to
the package.
As this is the first bit of TypeScript in this repo, the PR adds minimal
support for formatting `.d.ts` files with Prettier. It also opts out
`react-devtools-fusebox/dist/` from linting/formatting as a drive-by
fix.
For now, we'll just maintain the `.d.ts` file manually, but we could
consider leveraging
[`flow-api-translator`](https://www.npmjs.com/package/flow-api-translator)
to auto-generate it in the future.
## How did you test this change?
Build `react-devtools-fusebox`, observe that `dist/frontend.d.ts`
exists.
Following #28768, add a path to testing Fast JSX on www.
We want to measure the impact of Fast JSX and enable a path to testing
before string refs are completely removed in www (which is a work in
progress).
Without `disableStringRefs`, we need to copy any object with a `ref` key
so we can pass it through `coerceStringRef()` and copy it into the
object. This de-opt path is what is gated behind
`enableFastJSXWithStringRefs`.
The additional checks should have no perf impact in OSS as the flags
remain true there and the build output is not changed. For www, I've
benchmarked the addition of the boolean checks with values cached at
module scope. There is no significant change observed from our
benchmarks and any latency will apply to test and control branches
evenly. This added experiment complexity is temporary. We should be able
to clean it up, along with the flag checks for `enableRefAsProp` and
`disableStringRefs` shortly.
## Summary
This PR introduces a faster version of the `addProperties` function.
This new function is basically the `diffProperties` with `prevProps` set
to `null`, propagated constants, and all the unreachable code paths
collapsed.
## How did you test this change?
I've tested this change with [the benchmark
app](https://github.com/react-native-community/RNNewArchitectureApp/tree/new-architecture-benchmarks)
and got ~4.4% improvement in the view creation time.
When a React PR is opened CI will report large size changes. But for
critical packages like react-dom it reports always. In React 19 we moved
the build for react-dom the client reconciler from react-dom to
react-dom/client
This change adds react-dom-client artifacts for stable and oss channels
since that is originally what was being tracked. But since
react-dom/client always imports react-dom I left the original react-dom
packages as critical as well. They are small but it would be good to
keep an eye on them
To make a first time setup of the compiler truly config-less, default to
not compiling node_modules unless a user provided `sources` (advanced
option) is provided
ghstack-source-id: b0798052404d772ce6ee471e577699d4b0871d56
Pull Request resolved: https://github.com/facebook/react-forget/pull/2919
This uses the compiler runtime from `react/compiler-runtime` by default unless `compilerRuntime` is specifified in the Babel options which then imports the runtime from there. The `useMemoCache` hook is now named `c` in accordance with 4508873393
Unfortunately, I couldn't figure out how to import `react@beta` which already has that import as various react verstions were conflicting. If someone can figure this out it'd be fantastic. As a result, I had to update the default for the test runner to default the `compilerRuntime` option to `react` to preserve the previous behavior to import from `react`. Once upgraded to React 19, we should be able to remove that override.
Treat MethodCalls similar to general CallExpressions and mark them
as escaping in PruneNonEscapingScopes pass.
ghstack-source-id: 3c81bdb17f58fbeef8be24e7cb363172d1867217
Pull Request resolved: https://github.com/facebook/react-forget/pull/2925
Add a configurable list of known incompatible libraries.
Check all package.jsons for any uses of known incompatible libraries and
warn if found.
ghstack-source-id: 7329e3792b57458e681780cba3140a14a9b1a60d
Pull Request resolved: https://github.com/facebook/react-forget/pull/2923
Enables the Reanimated flag automatically if we find reanimated in the
user's list of plugins
ghstack-source-id: 20e83374612362a30d6c8cc7a903d9320e8cc23a
Pull Request resolved: https://github.com/facebook/react-forget/pull/2915
## Summary
I'm looking at cleaning up some unnecessary manual property flattening
in React Native and wanted to verify this behaviour is working as
expected, where properties from nested objects will always overwrite
properties from the base object.
## How did you test this change?
Unit tests
As discussed earlier, let's disable this validation pass by default for
oss since it still has many false positives
ghstack-source-id: fa3c21dde7cc4c3e4bb91dfa707e64bc7a9e088b
Pull Request resolved: https://github.com/facebook/react-forget/pull/2908
Rebasing and landing https://github.com/facebook/react/pull/28798
This PR was approved already but held back to give time for the sync.
Rebased and landing here without pushing to seb's remote to avoid
possibility of lost updates
---------
Co-authored-by: Sebastian Markbage <sebastian@calyptus.eu>
It turns out we already made refs writable in #25696, which has been in
canary for over a year. The approach in that PR also has the benefit of
being slightly more perf sensitive because it still uses a shared object
until the fiber is mounted. So let's just go back to that.
Add new reconciler methods since last breaking change to the README
based on usage and comments.
---------
Co-authored-by: Josh Story <josh.c.story@gmail.com>
This PR reorganizes the `react-dom` entrypoint to only pull in code that
is environment agnostic. Previously if you required anything from this
entrypoint in any environment the entire client reconciler was loaded.
In a prior release we added a server rendering stub which you could
alias in server environments to omit this unecessary code. After landing
this change this entrypoint should not load any environment specific
code.
While a few APIs are truly client (browser) only such as createRoot and
hydrateRoot many of the APIs you import from this package are only
useful in the browser but could concievably be imported in shared code
(components running in Fizz or shared components as part of an RSC app).
To avoid making these require opting into the client bundle we are
keeping them in the `react-dom` entrypoint and changing their
implementation so that in environments where they are not particularly
useful they do something benign and expected.
#### Removed APIs
The following APIs are being removed in the next major. Largely they
have all been deprecated already and are part of legacy rendering modes
where concurrent features of React are not available
* `render`
* `hydrate`
* `findDOMNode`
* `unmountComponentAtNode`
* `unstable_createEventHandle`
* `unstable_renderSubtreeIntoContainer`
* `unstable_runWithPrioirty`
#### moved Client APIs
These APIs were available on both `react-dom` (with a warning) and
`react-dom/client`. After this change they are only available on
`react-dom/client`
* `createRoot`
* `hydrateRoot`
#### retained APIs
These APIs still exist on the `react-dom` entrypoint but have normalized
behavior depending on which renderers are currently in scope
* `flushSync`: will execute the function (if provided) inside the
flushSync implemention of FlightServer, Fizz, and Fiber DOM renderers.
* `unstable_batchedUpdates`: This is a noop in concurrent mode because
it is now the only supported behavior because there is no legacy
rendering mode
* `createPortal`: This just produces an object. It can be called from
anywhere but since you will probably not have a handle on a DOM node to
pass to it it will likely warn in environments other than the browser
* preloading APIS such as `preload`: These methods will execute the
preload across all renderers currently in scope. Since we resolve the
Request object on the server using AsyncLocalStorage or the current
function stack in practice only one renderer should act upon the
preload.
In addition to these changes the server rendering stub now just rexports
everything from `react-dom`. In a future minor we will add a warning
when using the stub and in the next major we will remove the stub
altogether
This was probably a leftover from a previous time, but since this error
message throws when the dependency list is not an array literal, and not
just when its a rest spread, this PR updates the message to match.
ghstack-source-id: 28f2338212e56a67d3d477cea5abb6e9f3826488
Pull Request resolved: https://github.com/facebook/react-forget/pull/2902
This removes the automatic patching of the global `fetch` function in
Server Components environments to dedupe requests using `React.cache`, a
behavior that some RSC framework maintainers have objected to.
We may revisit this decision in the future, but for now it's not worth
the controversy.
Frameworks that have already shipped this behavior, like Next.js, can
reimplement it in userspace.
I considered keeping the implementation in the codebase and disabling it
by setting `enableFetchInstrumentation` to `false` everywhere, but since
that also disables the tests, it doesn't seem worth it because without
test coverage the behavior is likely to drift regardless. We can just
revert this PR later if desired.
Used this test scenario to clarify how callback refs work when detached
based on the availability of a cleanup function to update documentation
in https://github.com/reactjs/react.dev/pull/6770
Checking it in for additional test coverage and test-based documentation
It's not useful to output count of all failures,
as it's not actionable for the developer.
We'll still capture all failures in case we want
to add a rage option to this script.
ghstack-source-id: 4d5a1dd6a9616e6fd5e1166bb97fa047829b9273
Pull Request resolved: https://github.com/facebook/react-forget/pull/2889
Run the compiler on the globbed soruces.
The logger is used to capture the success and
failure compilation cases at the component level.
(If we were to compile the entire file directly,
we wouldn't get this granularity)
For now, we just log the number of success and
failures. In the future, we can provide a better
report building on this.
ghstack-source-id: 6d2d918190b6ed5d42b795491bbce29a950b9741
Pull Request resolved: https://github.com/facebook/react-forget/pull/2888
Exporting the hermes parser breaks the playground
as the hermes parser can not work in the browser.
No one is using this directly anyway -- snap and
others bundle hermes parser on their own, so,
let's remove it.
ghstack-source-id: d448c346eb137f8ba6ada4ad113e41a90b29baff
Pull Request resolved: https://github.com/facebook/react-forget/pull/2890
Use yargs to parse input of glob expression
matching the path of src files to compile.
ghstack-source-id: 6a35e958428cd08ef5c96e0014e072d3faf04064
Pull Request resolved: https://github.com/facebook/react-forget/pull/2886
This package will be used to check for violations
of the rules of react and other heuristics to
determine the health of a codebase.
The health of a codebase gives an expectation of
how easy it will be onboard on to the compiler.
ghstack-source-id: b52fc4e44f704e0544f15066d8905825a256dc4a
Pull Request resolved: https://github.com/facebook/react-forget/pull/2884
Stacked on #28849, #28854, #28853. Behind a flag.
If you're following along from the side-lines. This is probably not what
you think it is.
It's NOT a way to get updates to a component over time. The
AsyncIterable works like an Iterable already works in React which is how
an Array works. I.e. it's a list of children - not the value of a child
over time.
It also doesn't actually render one component at a time. The way it
works is more like awaiting the entire list to become an array and then
it shows up. Before that it suspends the parent.
To actually get these to display one at a time, you have to opt-in with
`<SuspenseList>` to describe how they should appear. That's really the
interesting part and that not implemented yet.
Additionally, since these are effectively Async Functions and uncached
promises, they're not actually fully "supported" on the client yet for
the same reason rendering plain Promises and Async Functions aren't.
They warn. It's only really useful when paired with RSC that produces
instrumented versions of these. Ideally we'd published instrumented
helpers to help with map/filter style operations that yield new
instrumented AsyncIterables.
The way the implementation works basically just relies on unwrapThenable
and otherwise works like a plain Iterator.
There is one quirk with these that are different than just promises. We
ask for a new iterator each time we rerender. This means that upon retry
we kick off another iteration which itself might kick off new requests
that block iterating further. To solve this and make it actually
efficient enough to use on the client we'd need to stash something like
a buffer of the previous iteration and maybe iterator on the iterable so
that we can continue where we left off or synchronously iterate if we've
seen it before. Similar to our `.value` convention on Promises.
In Fizz, I had to do a special case because when we render an iterator
child we don't actually rerender the parent again like we do in Fiber.
However, it's more efficient to just continue on where we left off by
reusing the entries from the thenable state from before in that case.
We have changed the shape (and the runtime) of React Elements. To help
avoid precompiled or inlined JSX having subtle breakages or deopting
hidden classes, I renamed the symbol so that we can early error if
private implementation details are used or mismatching versions are
used.
Why "transitional"? Well, because this is not the last time we'll change
the shape. This is just a stepping stone to removing the `ref` field on
the elements in the next version so we'll likely have to do it again.
Stacked on #28853 and #28854.
React supports rendering `Iterable` and will soon support
`AsyncIterable`. As long as it's multi-shot since during an update we
may have to rerender with new inputs an loop over the iterable again.
Therefore the `Iterator` and `AsyncIterator` types are not supported
directly as a child of React - and really it shouldn't pass between
Hooks or components neither for this reason. For parity, that's also the
case when used in Server Components.
However, there is a special case when the component rendered itself is a
generator function. While it returns as a child an `Iterator`, the React
Element itself can act as an `Iterable` because we can re-evaluate the
function to create a new generator whenever we need to.
It's also very convenient to use generator functions over constructing
an `AsyncIterable`. So this is a proposal to special case the
`Generator`/`AsyncGenerator` returned by a (Async) Generator Function.
In Flight this means that when we render a Server Component we can
serialize this value as an `Iterable`/`AsyncIterable` since that's
effectively what rendering it on the server reduces down to. That way if
Fiber can receive the result in any position.
For SuspenseList this would also need another special case because the
children of SuspenseList represent "rows".
`<SuspenseList><Component /></SuspenseList>` currently is a single "row"
even if the component renders multiple children or is an iterator. This
is currently different if Component is a Server Component because it'll
reduce down to an array/AsyncIterable and therefore be treated as one
row per its child. This is different from `<SuspenseList><Component
/><Component /></SuspenseList>` since that has a wrapper array and so
this is always two rows.
It probably makes sense to special case a single-element child in
`SuspenseList` to represent a component that generates rows. That way
you can use an `AsyncGeneratorFunction` to do this.
For [`AsyncIterable`](https://github.com/facebook/react/pull/28847) we
encode `AsyncIterator` as a separate tag.
Previously we encoded `Iterator` as just an Array. This adds a special
encoding for this. Technically this is a breaking change.
This is kind of an edge case that you'd care about the difference but it
becomes more important to treat these correctly for the warnings here
#28853.
This doesn't change production behavior. We always render Iterables to
our best effort in prod even if they're Iterators.
But this does change the DEV warnings which indicates which are valid
patterns to use.
It's a footgun to use an Iterator as a prop when you pass between
components because if an intermediate component rerenders without its
parent, React won't be able to iterate it again to reconcile and any
mappers won't be able to re-apply. This is actually typically not a
problem when passed only to React host components but as a pattern it's
a problem for composability.
We used to warn only for Generators - i.e. Iterators returned from
Generator functions. This adds a warning for Iterators created by other
means too (e.g. Flight or the native Iterator utils). The heuristic is
to check whether the Iterator is the same as the Iterable because that
means it's not possible to get new iterators out of it. This case used
to just yield non-sense like empty sets in DEV but not in prod.
However, a new realization is that when the Component itself is a
Generator Function, it's not actually a problem. That's because the
React Element itself works as an Iterable since we can ask for new
generators by calling the function again. So this adds a special case to
allow the Generator returned from a Generator Function's direct child.
The principle is “don’t pass iterators around” but in this case there is
no iterator floating around because it’s between React and the JS VM.
Also see #28849 for context on AsyncIterables.
Related to this, but Hooks should ideally be banned in these for the
same reason they're banned in Async Functions.
This disables symbol renaming in production builds. The original
variable and function names are preserved. All other forms of
compression applied by Closure (dead code elimination, inlining, etc)
are unchanged — the final program is identical to what we were producing
before, just in a more readable form.
The motivation is to make it easier to debug React issues that only
occur in production — the same reason we decided to start shipping
sourcemaps in #28827 and #28827.
However, because most apps run their own minification step on their npm
dependencies, it's not necessary for us to minify the symbols before
publishing — it'll be handled the app, if desired.
This is the same strategy Meta has used to ship React for years. The
React build itself has unminified symbols, but they get minified as part
of Meta's regular build pipeline.
Even if an app does not minify their npm dependencies, gzip covers most
of the cost of symbol renaming anyway.
This saves us from having to ship sourcemaps, which means even apps that
don't have sourcemaps configured will be able to debug the React build
as easily as they would any other npm dependency.
Adds an experimental feature flag to the implementation of useMemoCache,
the internal cache used by the React Compiler (Forget).
When enabled, instead of treating the cache as copy-on-write, like we do
with fibers, we share the same cache instance across all render
attempts, even if the component is interrupted before it commits.
If an update is interrupted, either because it suspended or because of
another update, we can reuse the memoized computations from the previous
attempt. We can do this because the React Compiler performs atomic
writes to the memo cache, i.e. it will not record the inputs to a
memoization without also recording its output.
This gives us a form of "resuming" within components and hooks.
This only works when updating a component that already mounted. It has
no impact during initial render, because the memo cache is stored on the
fiber, and since we have not implemented resuming for fibers, it's
always a fresh memo cache, anyway.
However, this alone is pretty useful — it happens whenever you update
the UI with fresh data after a mutation/action, which is extremely
common in a Suspense-driven (e.g. RSC or Relay) app.
So the impact of this feature is faster data mutations/actions (when the
React Compiler is used).
Meta uses various tools built on top of the "react-reconciler" package
but that package needs to match the version of the "react" package.
This means that it should be synced at the same time. However, more than
that the feature flags between the "react" package and the
"react-reconciler" package needs to line up. Since FB has custom feature
flags, it can't use the OSS version of react-reconciler.
In #26446 we started publishing non-minified versions of our production
build artifacts, along with source maps, for easier debugging of React
when running in production mode.
The way it's currently set up is that these builds are generated
*before* Closure compiler has run. Which means it's missing many of the
optimizations that are in the final build, like dead code elimination.
This PR changes the build process to run Closure on the non-minified
production builds, too, by moving the sourcemap generation to later in
the pipeline.
The non-minified builds will still preserve the original symbol names,
and we'll use Prettier to add back whitespace. This is the exact same
approach we've been using for years to generate production builds for
Meta.
The idea is that the only difference between the minified and non-
minified builds is whitespace and symbol mangling. The semantic
structure of the program should be identical.
To implement this, I disabled symbol mangling when running Closure
compiler. Then, in a later step, the symbols are mangled by Terser. This
is when the source maps are generated.
Stacked on #28872
renderToStaticNodeStream was not originally deprecated when
renderToNodeStream was deprecated because it did not yet have a clear
analog in the modern streaming implementation for SSR. In React 19 we
have already removed renderToNodeStream. This change removes
renderToStaticNodeStream as well because you can replicate it's
semantics using renderToPipeableStream with onAllReady or
renderToReadableStream with await stream.allready.
This commit adds warnings indicating that `renderToStaticNodeStream`
will be removed in an upcoming React release. This API has been legacy,
is not widely used (renderToStaticMarkup is more common) and has
semantically eqiuvalent implementations with renderToReadableStream and
renderToPipeableStream.
stacked on #28870
inline script children have been encoded as HTML for a while now but
this can easily break script parsing so practically if you were
rendering inline scripts you were using dangerouslySetInnerHTML. This is
not great because now there is no escaping at all so you have to be even
more careful. While care should always be taken when rendering untrusted
script content driving users to use dangerous APIs is not the right
approach and in this PR the escaping functionality used for
bootstrapScripts and importMaps is being extended to any inline script.
the approach is to escape 's' or 'S" with the appropriate unicode code
point if it is inside a <script or </script sequence. This has the nice
benefit of minimally escaping the text for readability while still
preserving full js parsing capabilities. As articulated when we
introduced this escaping for prior use cases this is only safe because
we are escaping the entire script content. It would be unsafe if we were
not escaping the entirety of the script because we would no longer be
able to ensure there are no earlier or later <script sequences that put
the parser in unexpected states.
style text content has historically been escaped as HTML which is
non-sensical and often leads users to using dangerouslySetInnerHTML as a
matter of course. While rendering untrusted style rules is a security
risk React doesn't really provide any special protection here and
forcing users to use a completely unescaped API is if anything worse. So
this PR updates the style escaping rules for Fizz to only escape the
text content to ensure the tag scope cannot be closed early. This is
accomplished by encoding "s" and "S" as hexadecimal unicode
representation "\73 " and "\53 " respectively when found within a
sequence like </style>. We have to be careful to support casing here
just like with the script closing tag regex for bootstrap scripts.
## Summary
The Forget codename needs to be hidden from the UI to avoid confusion.
Going forward, we'll be referring to this set of features as part of the
larger React compiler. We'll be describing the primary feature that
we've built so far as auto-memoization, and this badge helps devs see
which components have been automatically memoized by the compiler.
## How did you test this change?
- force Forget badge on with and without the presence of other badges
- confirm colors/UI in light and dark modes
- force badges on for `ElementBadges`, `InspectableElementBadges`,
`IndexableElementBadges`
- Running yarn start in packages/react-devtools-shell
[demo
video](https://github.com/facebook/react/assets/973058/fa829018-7644-4425-8395-c5cd84691f3c)
For fbsource we've historically used a separate repo for imports due to
internal limitations in Diff Train. Those have been lifted so we can now
commit this branch here and then we can import from this repo (and get
rid of the other repo)
Previously, the `refs` property of a class component instance was
read-only by user code — only React could write to it, and until/unless
a string ref was used, it pointed to a shared empty object that was
frozen in dev to prevent userspace mutations.
Because string refs are deprecated, we want users to be able to codemod
all their string refs to callback refs. The safest way to do this is to
output a callback ref that assigns to `this.refs`.
So to support this, we need to make `this.refs` writable by userspace.
## Summary
This PR adds early return to the `diff` function. We don't need to go
through all the entries of `nextProps`, process and deep-diff the values
if `nextProps` is the same object as `prevProps`. Roughly 6% of all
`diffProperties` calls can be skipped.
## How did you test this change?
RNTester.
<!--
Thanks for submitting a pull request!
We appreciate you spending the time to work on these changes. Please
provide enough information so that others can review your pull request.
The three fields below are mandatory.
Before submitting a pull request, please make sure the following is
done:
1. Fork [the repository](https://github.com/facebook/react) and create
your branch from `main`.
2. Run `yarn` in the repository root.
3. If you've fixed a bug or added code that should be tested, add tests!
4. Ensure the test suite passes (`yarn test`). Tip: `yarn test --watch
TestName` is helpful in development.
5. Run `yarn test --prod` to test in the production environment. It
supports the same options as `yarn test`.
6. If you need a debugger, run `yarn test --debug --watch TestName`,
open `chrome://inspect`, and press "Inspect".
7. Format your code with
[prettier](https://github.com/prettier/prettier) (`yarn prettier`).
8. Make sure your code lints (`yarn lint`). Tip: `yarn linc` to only
check changed files.
9. Run the [Flow](https://flowtype.org/) type checks (`yarn flow`).
10. If you haven't already, complete the CLA.
Learn more about contributing:
https://reactjs.org/docs/how-to-contribute.html
-->
## Summary
<!--
Explain the **motivation** for making this change. What existing problem
does the pull request solve?
-->
## How did you test this change?
<!--
Demonstrate the code is solid. Example: The exact commands you ran and
their output, screenshots / videos if the pull request changes the user
interface.
How exactly did you verify that your PR solves the issue you wanted to
solve?
If you leave this empty, your PR will very likely be closed.
-->
## Summary
This pull request converts the CircleCI workflows to GitHub actions
workflows. [Github Actions
Importer](https://github.com/github/gh-actions-importer) was used to
convert the workflows initially, then I edited them manually to correct
errors in translation.
**Issues**
1. facebook/react/devtools_regression_tests
The scripts that this workflow calls need to be modified.
## How did you test this change?
I tested these changes in a forked repo. You can [view the logs of this
workflow in my fork](https://github.com/robandpdx/react/actions).
https://fburl.com/workplace/f6mz6tmw
In React 19 React will finally stop publishing UMD builds. This is
motivated primarily by the lack of use of UMD format and the added
complexity of maintaining build infra for these releases. Additionally
with ESM becoming more prevalent in browsers and services like esm.sh
which can host React as an ESM module there are other options for doing
script tag based react loading.
This PR removes all the UMD build configs and forks.
There are some fixtures that still have references to UMD builds however
many of them already do not work (for instance they are using legacy
features like ReactDOM.render) and rather than block the removal on
these fixtures being brought up to date we'll just move forward and fix
or removes fixtures as necessary in the future.
Fixes issue where if the first letter of the expected string appeared
anywhere in actual message, the assertion would pass, leading to false
negatives. We should check the entire expected string.
---------
Co-authored-by: Ricky <rickhanlonii@gmail.com>
This wasn't clearly articulated and tested why the code structure is
like this but I think the logic is correct - or at least consistent with
the weird semantics.
We place this top-level fragment check inside the recursion so that you
can resolve how many every Lazy or Usable wrappers you want and it still
preserves the same semantics if they weren't there (which they might not
be as a matter of a race condition).
However, we don't actually recurse with the top-level fragment
unwrapping itself because nesting a bunch of keyless fragments isn't the
same as a single fragment/element.
So that when we end up referring to it in more places, it's only one.
We don't do this same pattern for regular `Symbol.iterator` because we
also support the string `"@@iterator"` for backwards compatibility.
This adds support in Flight for serializing four kinds of streams:
- `ReadableStream` with objects as a model. This is a single shot
iterator so you can read it only once. It can contain any value
including Server Components. Chunks are encoded as is so if you send in
10 typed arrays, you get the same typed arrays out on the other side.
- Binary `ReadableStream` with `type: 'bytes'` option. This supports the
BYOB protocol. In this mode, the receiving side just gets `Uint8Array`s
and they can be split across any single byte boundary into arbitrary
chunks.
- `AsyncIterable` where the `AsyncIterator` function is different than
the `AsyncIterable` itself. In this case we assume that this might be a
multi-shot iterable and so we buffer its value and you can iterate it
multiple times on the other side. We support the `return` value as a
value in the single completion slot, but you can't pass values in
`next()`. If you want single-shot, return the AsyncIterator instead.
- `AsyncIterator`. These gets serialized as a single-shot as it's just
an iterator.
`AsyncIterable`/`AsyncIterator` yield Promises that are instrumented
with our `.status`/`.value` convention so that they can be synchronously
looped over if available. They are also lazily parsed upon read.
We can't do this with `ReadableStream` because we use the native
implementation of `ReadableStream` which owns the promises.
The format is a leading row that indicates which type of stream it is.
Then a new row with the same ID is emitted for every chunk. Followed by
either an error or close row.
`AsyncIterable`s can also be returned as children of Server Components
and then they're conceptually the same as fragment arrays/iterables.
They can't actually be used as children in Fizz/Fiber but there's a
separate plan for that. Only `AsyncIterable` not `AsyncIterator` will be
valid as children - just like sync `Iterable` is already supported but
single-shot `Iterator` is not. Notably, neither of these streams
represent updates over time to a value. They represent multiple values
in a list.
When the server stream is aborted we also close the underlying stream.
However, closing a stream on the client, doesn't close the underlying
stream.
A couple of possible follow ups I'm not planning on doing right now:
- [ ] Free memory by releasing the buffer if an Iterator has been
exhausted. Single shots could be optimized further to release individual
items as you go.
- [ ] We could clean up the underlying stream if the only pending data
that's still flowing is from streams and all the streams have cleaned
up. It's not very reliable though. It's better to do cancellation for
the whole stream - e.g. at the framework level.
- [ ] Implement smarter Binary Stream chunk handling. Currently we wait
until we've received a whole row for binary chunks and copy them into
consecutive memory. We need this to preserve semantics when passing
typed arrays. However, for binary streams we don't need that. We can
just send whatever pieces we have so far.
Per team discussion, this upgrades the `initialValue` argument for
`useDeferredValue` from experimental to canary.
- Original implementation PR:
https://github.com/facebook/react/pull/27500
- API documentation PR: https://github.com/reactjs/react.dev/pull/6747
I left it disabled at Meta for now in case there's old code somewhere
that is still passing an `options` object as the second argument.
This allows the plugin to be configured to run on an allowlist, rather
than compiling all files helping with an incremental rollout plan.
The sources option takes both an array of path strings or a function
to be flexible.
For now I've left this be optional but we can make it required.
ghstack-source-id: 282a33dc8d08d47f699894692e0fcc813dff5b77
Pull Request resolved: https://github.com/facebook/react-forget/pull/2855
No need to commit this in the repo, but keeping this locally helps with
building the feedback repo.
ghstack-source-id: 28f08992b1ab856567a5338af07e12ac820a7298
Pull Request resolved: https://github.com/facebook/react-forget/pull/2854
With the enableBinaryFlight flag on we should encode typed arrays and
blobs in the Reply direction too for parity.
It's already possible to pass Blobs inside FormData but you should be
able to pass them inside objects too.
We encode typed arrays as blobs and then unwrap them automatically to
the right typed array type.
Unlike the other protocol, I encode the type as a reference tag instead
of row tag. Therefore I need to rename the tags to avoid conflicts with
other tags in references. We are running out of characters though.
This hasn't been updated in a long time, and it getting really large.
You can view the authors locally via the git history with:
``
git shortlog -se | perl -spe 's/^\s+\d+\s+//'
``
## Overview
There's currently a bug in RN now that we no longer re-throw errors. The
`showErrorDialog` function in React Native only logs the errors as soft
errors, and never a fatal. RN was depending on the global handler for
the fatal error handling and logging.
Instead of fixing this in `ReactFiberErrorDialog`, we can implement the
new root options in RN to handle caught/uncaught/recoverable in the
respective functions, and delete ReactFiberErrorDialog. I'll follow up
with a RN PR to implement these options and fix the error handling.
<!--
Thanks for submitting a pull request!
We appreciate you spending the time to work on these changes. Please
provide enough information so that others can review your pull request.
The three fields below are mandatory.
Before submitting a pull request, please make sure the following is
done:
1. Fork [the repository](https://github.com/facebook/react) and create
your branch from `main`.
2. Run `yarn` in the repository root.
3. If you've fixed a bug or added code that should be tested, add tests!
4. Ensure the test suite passes (`yarn test`). Tip: `yarn test --watch
TestName` is helpful in development.
5. Run `yarn test --prod` to test in the production environment. It
supports the same options as `yarn test`.
6. If you need a debugger, run `yarn test --debug --watch TestName`,
open `chrome://inspect`, and press "Inspect".
7. Format your code with
[prettier](https://github.com/prettier/prettier) (`yarn prettier`).
8. Make sure your code lints (`yarn lint`). Tip: `yarn linc` to only
check changed files.
9. Run the [Flow](https://flowtype.org/) type checks (`yarn flow`).
10. If you haven't already, complete the CLA.
Learn more about contributing:
https://reactjs.org/docs/how-to-contribute.html
-->
## Summary
<!--
Explain the **motivation** for making this change. What existing problem
does the pull request solve?
-->
The ReadableStreamController for [direct
streams](https://bun.sh/docs/api/streams#direct-readablestream) in Bun
supports a flush() method to flush all buffered items to its underlying
sink.
Without manually calling flush(), all buffered items are only flushed to
the underlying sink when the stream is closed. This behavior causes the
shell rendered against Suspense boundaries never to be flushed to the
underlying sink.
## How did you test this change?
<!--
Demonstrate the code is solid. Example: The exact commands you ran and
their output, screenshots / videos if the pull request changes the user
interface.
How exactly did you verify that your PR solves the issue you wanted to
solve?
If you leave this empty, your PR will very likely be closed.
-->
A lot of changes to the test runner will need to be made in order to
support the Bun runtime. A separate test was manually run in order to
ensure that the changes made are correct.
The test works by sanity-checking that the shell rendered against
Suspense boundaries are emitted first in the stream.
This test was written and run on Bun v1.1.3.
```ts
import { Suspense } from "react";
import { renderToReadableStream } from "react-dom/server";
if (!import.meta.resolveSync("react-dom/server").endsWith("server.bun.js")) {
throw new Error("react-dom/server is not the correct version:\n " + import.meta.resolveSync("react-dom/server"));
}
const A = async () => {
await new Promise(resolve => setImmediate(resolve));
return <div>hi</div>;
};
const B = async () => {
return (
<Suspense fallback={<div>loading</div>}>
<A />
</Suspense>
);
};
const stream = await renderToReadableStream(<B />);
let text = "";
let count = 0;
for await (const chunk of stream) {
text += new TextDecoder().decode(chunk);
count++;
}
if (
text !==
`<!--$?--><template id="B:0"></template><div>loading</div><!--/$--><div hidden id="S:0"><div>hi</div></div><script>$RC=function(b,c,e){c=document.getElementById(c);c.parentNode.removeChild(c);var a=document.getElementById(b);if(a){b=a.previousSibling;if(e)b.data="$!",a.setAttribute("data-dgst",e);else{e=b.parentNode;a=b.nextSibling;var f=0;do{if(a&&8===a.nodeType){var d=a.data;if("/$"===d)if(0===f)break;else f--;else"$"!==d&&"$?"!==d&&"$!"!==d||f++}d=a.nextSibling;e.removeChild(a);a=d}while(a);for(;c.firstChild;)e.insertBefore(c.firstChild,a);b.data="$"}b._reactRetry&&b._reactRetry()}};$RC("B:0","S:0")</script>`
) {
throw new Error("unexpected output");
}
if (count !== 2) {
throw new Error("expected 2 chunks from react ssr stream");
}
```
The compiler has an optimisation where it transforms a simple arrow
function with only a return statement to a implicit arrow function.
In the case, there's a directive in this simple arrow function, the
directive gets dropped.
Instead of dropping the directive, the compiler should perform this
optimisation only if there are no directives.
ghstack-source-id: 514cd2440025986a2d6d950694a7339d779b09f2
Pull Request resolved: https://github.com/facebook/react-forget/pull/2848
The useMemoCache polyfill doesn't have access to the fiber, and it
simply uses state, which does not work with the existing devtools
badge for the compiler.
With this PR, devtools will look on the very first hook's state for the
memo cache sentinel and display the Forget badge if present.
The polyfill will add this sentinel to it's state (the cache array).
The flight-browser fixtures doesn't make sense. It also uses UMD builds
which are being removed so we'd have to make it use esm.sh or something
and really it just won't work because it needs to be built by webpack to
not error. We could potentially shim the webpack globals but really the
right thing is to publish the esm version of react-server and use esm.sh
to load a browser only esm demo of react-server. This change removes the
flight-browser fixture
Support propagating theme from Chrome DevTools frontend, the field is
optional.
Next step, which is out of scope of this project and general improvement
for React DevTools: teach RDT to listen to theme changes and if the
theme preference is set to `auto` in settings, update the theme
accordingly with the browser devtools.
To make the polyfill work well with devtools, we add this sentinel.
Devtools will look for this sentinel before adding the Forget badge.
ghstack-source-id: d246040f67da48fa46f818753c8bf26dff56f390
Pull Request resolved: https://github.com/facebook/react-forget/pull/2847
## Summary
Stacked on https://github.com/facebook/react/pull/28552. Review only the
[last commit at the
top](c69952f1bf).
These changes add new package `react-devtools-fusebox`, which is the
entrypoint for the RDT Frontend, which will be used in Chrome DevTools
panel. The main differences from other frontend shells (extension,
standalone) are:
1. This package builds scripts in ESM format, this is required by Chrome
DevTools, see webpack config:
c69952f1bf/packages/react-devtools-fusebox/webpack.config.frontend.js (L50-L52)
2. The build includes styles in a separate `.css` file, which is
required for Chrome DevTools: styles are loaded lazily once panel is
mounted.
## Summary
RDT backend will now expose method `connectWithCustomMessagingProtocol`,
which will be similar to the classic `connectToDevTools` one, but with
few differences:
1. It delegates the communication management between frontend and
backend to the owner (whos injecting RDT backend). Unlike the
`connectToDevTools`, which is relying on websocket connection and
receives host and port as an arguments.
2. It returns a callback, which can be used for unsubscribing the
current backend instance from the global DevTools hook.
This is a prerequisite for any non-browser RDT integration, which is not
designed to be based on websocket.
Hoistables should never flush before the preamble however there is a
surprisingly easy way to trigger this to happen by suspending in the
shell of the app. This change modifies the flushing behavior to not emit
any hoistables before the preamble has written. It accomplishes this by
aborting the flush early if there are any pending root tasks remaining.
It's unfortunate we need this extra condition but it's essential that we
don't emit anything before the preamble and at the moment I don't see a
way to do that without introducing a new condition.
There is a test that began to fail with this update. It turns out that
in node the root can be blocked during a resume even for a component
inside a Suspense boundary if that boundary was part of the prerender.
This means that with the current heuristic in this PR boundaries cannot
be flushed during resume until the root is unblocked. This is not ideal
but this is already how Edge works because the root blocks the stream in
that case. This just makes Node deopt in a similar way to edge. We
should improve this but we ought to do so in a way that works for edge
too and it needs to be more comprehensive.
## Overview
**Internal React repo tests only**
Depends on https://github.com/facebook/react/pull/28710
Adds three new assertions:
- `assertConsoleLogDev`
- `assertConsoleWarnDev`
- `assertConsoleErrorDev`
These will replace this pattern:
```js
await expect(async () => {
await expect(async () => {
await act(() => {
root.render(<Fail />)
});
}).toThrow();
}).toWarnDev('Warning');
```
With this:
```js
await expect(async () => {
await act(() => {
root.render(<Fail />)
});
}).toThrow();
assertConsoleWarnDev('Warning');
```
It works similar to our other `assertLog` matchers which clear the log
and assert on it, failing the tests if the log is not asserted before
the test ends.
## Diffs
There are a few improvements I also added including better log diffs and
more logging.
When there's a failure, the output will look something like:
<img width="655" alt="Screenshot 2024-04-03 at 11 50 08 AM"
src="https://github.com/facebook/react/assets/2440089/0c4bf1b2-5f63-4204-8af3-09e0c2d752ad">
Check out the test suite for snapshots of all the failures we may log.
Based on:
- #28808
- #28804
---
This adds a React DOM method called requestFormReset that schedules a
form reset to occur when the current transition completes.
Internally, it's the same method that's called automatically whenever a
form action is submitted. It only affects uncontrolled form inputs. See
https://github.com/facebook/react/pull/28804 for details.
The reason for the public API is so UI libraries can implement their own
action-based APIs and maintain the form-resetting behavior, something
like this:
```js
function onSubmit(event) {
// Disable default form submission behavior
event.preventDefault();
const form = event.target;
startTransition(async () => {
// Request the form to reset once the action
// has completed
requestFormReset(form);
// Call the user-provided action prop
await action(new FormData(form));
})
}
```
Based on:
- #28804
---
This sets adds a new ReactDOM export called requestFormReset, including
setting up the export and creating a method on the internal ReactDOM
dispatcher. It does not yet add any implementation.
Doing this in its own commit for review purposes.
The API itself will be explained in the next PR.
This updates the behavior of form actions to automatically reset the
form's uncontrolled inputs after the action finishes.
This is a frequent feature request for people using actions and it
aligns the behavior of client-side form submissions more closely with
MPA form submissions.
It has no impact on controlled form inputs. It's the same as if you
called `form.reset()` manually, except React handles the timing of when
the reset happens, which is tricky/impossible to get exactly right in
userspace.
The reset shouldn't happen until the UI has updated with the result of
the action. So, resetting inside the action is too early.
Resetting in `useEffect` is better, but it's later than ideal because
any effects that run before it will observe the state of the form before
it's been reset.
It needs to happen in the mutation phase of the transition. More
specifically, after all the DOM mutations caused by the transition have
been applied. That way the `defaultValue` of the inputs are updated
before the values are reset. The idea is that the `defaultValue`
represents the current, canonical value sent by the server.
Note: this change has no effect on form submissions that aren't
triggered by an action.
`<noscript>` scopes should be considered inert from the perspective of
Fizz since we assume they'll only be used in rare and adverse
circumstances. When we added preload support for img tags we did not
include the noscript scope check in the opt-out for preloading. This
change adds it in
fixes: #27910
Fixes a tiny inconsistency with compiler options where one was all
uppercase and one all lowercase by normalizing to lowercase regardless
of the casing of the user's config.
ghstack-source-id: fe60a3259de89a1b3fdd7475950e16e96cc57f6b
Pull Request resolved: https://github.com/facebook/react-forget/pull/2832
It was never clear to me what the difference between Wipe and Reset was.
Let's just get rid of one and reset to something more useful instead of
fibonacci.
ghstack-source-id: 4f88a1c1da2d0fd9e1f26e4859c12db0fc961af2
Pull Request resolved: https://github.com/facebook/react-forget/pull/2837
This compresses more efficiently than the base64 encoding we were
previously using, which makes sharing URLs a little less unwieldy and
takes up less space in local storage. Using
some real code as an example, lz-string compresses to 8040 bytes,
whereas the original base64 encoding we were using compresses to 16504
bytes
ghstack-source-id: b8f1089889b94b07d6f419606b798ffddb8863ba
Pull Request resolved: https://github.com/facebook/react-forget/pull/2834
These test don't `assertLog` or `waitFor` so we don't need to
`Scheduler.log`. Ideally we would, but since they're fuzzers it's a bit
difficult to know what the expected log is from the helper.
Since this doesn't regress current test behavior, we can improve them
after this to unblock https://github.com/facebook/react/pull/28737
We want to warn if we detect that an app is using an outdated JSX
transform. We can't just warn if `createElement` is called because we
still support `createElement` when it's called manually. We only want to
warn if `createElement` is output by the compiler.
The heuristic is to check for a `__self` prop, which is an optional,
internal prop that older transforms used to pass to `createElement` for
better debugging in development mode.
If `__self` is present, we `console.warn` once with advice to upgrade to
the modern JSX transform. Subsequent elements will not warn.
There's a special case we have to account for: when a static "key" prop
is defined _after_ a spread, the modern JSX transform outputs
`createElement` instead of `jsx`. (This is because with `jsx`, a spread
key always takes precedence over a static key, regardless of the order,
whereas `createElement` respects the order.) To avoid a false positive
warning, we skip the warning whenever a `key` prop is present.
Fixes a bug that happens when an error occurs during hydration, React
switches to client rendering, and then the client render suspends. It
works correctly if there's a Suspense boundary on the stack, but not if
it happens in the shell of the app.
Prior to this fix, the app would crash with an "Unknown root exit
status" error.
I left a TODO comment for how we might refactor this code to be less
confusing in the future.
Fix for an issue introduced in #28473 where cloneElement() with a string
ref fails due to lack of an owner. We should use the current owner in
this case.
---------
Co-authored-by: Rick Hanlon <rickhanlonii@fb.com>
Previously if the external runtime was enabled Fizz tests would use it
exclusively. However now that this flag is enabled for OSS and Meta
builds this means we were no longer testing the inline script runtime.
This changes the test flags to produce some runs where we test the
inline script runtime and others where we test the external runtime
the external runtime will be tested if the flag is enabled and
* Meta Builds: variant is true
* OSS Builds: experiemental is true
this gives us decent coverage. long term we should probably bring
variant to OSS builds since we will eventually want to test both modes
even when the external runtime is stable.
When packaging we want to infer that a bundle exists for a
`react-server` file even if it isn't explicitly configured. This is
useful in particular for the react-server entrypoints that error on
import that were recently added to `react-dom`
This change also cleans up a wayward comment left behind in a prior PR
Follow up to #28783 and #28786.
Since we've changed the implementations of these we can rename them to
something a bit more descriptive while we're at it, since anyone
depending on them will need to upgrade their code anyway.
"react" with no condition:
`__CLIENT_INTERNALS_DO_NOT_USE_OR_WARN_USERS_THEY_CANNOT_UPGRADE`
"react" with "react-server" condition:
`__SERVER_INTERNALS_DO_NOT_USE_OR_WARN_USERS_THEY_CANNOT_UPGRADE`
"react-dom":
`__DOM_INTERNALS_DO_NOT_USE_OR_WARN_USERS_THEY_CANNOT_UPGRADE`
We have a different set of dispatchers that Flight uses. This also
includes the `jsx-runtime` which must also be aliased to use the right
version.
To ensure the right versions are used together we rename the export of
the SharedInternals from 'react' and alias it in relevant bundles.
This is similar to #28771 but for isomorphic. We need a make over for
these dispatchers anyway so this is the first step. Also helps flush out
some internals usage that will break anyway.
It flattens the inner mutable objects onto the ReactSharedInternals.
`react-server` precludes loading code that expects to be run in a client
context. This includes react-dom/client react-dom/server
react-dom/unstable_testing react-dom/profiling and react-dom/static
This update makes importing any of these client only entrypoints an
error
Treat async (boolean prop) consistently with Float. Previously float
checked if `props.async === true` (or not true) but the rest of
react-dom considers anything truthy that isn't a function or symbol as
`true`. This PR normalizes the Float behavior.
Stacked on #28751
Historically explicit hydration scheduling used the reconciler's update
priority to schedule the hydration. There was a lingering todo to switch
to using event priority in the absence of an explicit update priority.
This change updates the hydration priority by referring to the event
priority if no update priority is set
Stacked on #28771
ReactDOMCurrentDispatcher has longer property names for various methods.
These methods are only ever called internally and don't need to be
represented with as many characters. This change shortens the names and
aligns them with the hint codes we use in Flight. This alignment is
passive since not all dispatcher methods will exist as flight
instructions but where they can line up it seems reasonable to make them
do so
Stacked on #28751
ReactDOMSharedInternals uses properties of considerable length to model
mutuable state. These properties are not mangled during minification and
contribute a not insigificant amount to the uncompressed bundle size and
to a lesser degree compressed bundle size.
This change rewrites the DOMInternals in a way that shortens property
names so we can have smaller builds.
It also treats the entire object as a mutable container rather than
having different mutable sub objects.
The same treatment should be given to ReactSharedInternals
We used to assume that outlined models are emitted before the reference
(which was true before Blobs). However, it still wasn't safe to assume
that all the data will be available because an "import" (client
reference) can be async and therefore if it's directly a child of an
outlined model, it won't be able to update in place.
This is a similar problem as the one hit by @unstubbable in #28669 with
elements, but a little different since these don't follow the same way
of wrapping.
I don't love the structuring of this code which now needs to pass a
first class mapper instead of just being known code. It also shares the
host path which is just an identity function. It wouldn't necessarily
pass my own review but I don't have a better one for now. I'd really
prefer if this was done at a "row" level but that ends up creating even
more code.
Add test for Blob in FormData and async modules in Maps.
We landed a flag to disable test utils in many builds but we need to
fork the entrypoint to make it work with tests properly. This also
removes test-utils implementations from builds that do not support it.
Currently in OSS builds the only thing in test-utils is a reexport of
`act`
## Summary
1. RDT browser extension's content scripts will now ship source maps
(without source in prod, to save some bundle size).
2. `installHook` content script will be ignore listed via `ignoreList`
field in the corresponding source map.
3. Previously, source map for backend file used `x_google_ignoreList`
naming, now `ignoreList`.
## How did you test this change?
1. `ignoreList-test.js`
2. Tested manually that I don't see `installHook` in stack traces when
`console.error` is called.
This PR moves `flushSync` out of the reconciler. there is still an
internal implementation that is used when these semantics are needed for
React methods such as `unmount` on roots.
This new isomorphic `flushSync` is only used in builds that no longer
support legacy mode.
Additionally all the internal uses of flushSync in the reconciler have
been replaced with more direct methods. There is a new
`updateContainerSync` method which updates a container but forces it to
the Sync lane and flushes passive effects if necessary. This combined
with flushSyncWork can be used to replace flushSync for all instances of
internal usage.
We still maintain the original flushSync implementation as
`flushSyncFromReconciler` because it will be used as the flushSync
implementation for FB builds. This is because it has special legacy mode
handling that the new isomorphic implementation does not need to
consider. It will be removed from production OSS builds by closure
though
Currently updatePriority is tracked in the reconciler. `flushSync` is
going to be implemented reconciler agnostic soon and we need to move the
tracking of this state to the renderer and out of reconciler. This
change implements new renderer bin dings for getCurrentUpdatePriority
and setCurrentUpdatePriority.
I was originally going to have the getter also do the event priority
defaulting using window.event so we eliminate getCur rentEventPriority
but this makes all the callsites where we store the true current
updatePriority on the stack harder to work with so for now they remain
separate.
I also moved runWithPriority to the renderer since it really belongs
whereever the state is being managed and it is only currently exposed in
the DOM renderer.
Additionally the current update priority is not stored on
ReactDOMSharedInternals. While not particularly meaningful in this
change it opens the door to implementing `flushSync` outside of the
reconciler
Follow up to #28768.
The modern JSX runtime (`jsx`) does not need to check if each prop is a
direct property with `hasOwnProperty` because the compiler always passes
a plain object.
I'll leave the check in the old JSX runtime (`createElement`) since that
one can be called manually with any kind of object, and if there were
old user code that relied on this for some reason, it would be using
that runtime.
CannotPreserveMemoization
We do need to fix the error location to point to the "callsite" rather
than the definition of the useMemo callback, but that aside, even if the
error message were perfect, it's not meant to be actionable to the user.
So let's change the severity to CannotPreserveMemoization. This
preserves the validation, but the eslint plugin won't report it.
ghstack-source-id: 722c88922884de05e89030a7b001bd93e0a2a114
Pull Request resolved: https://github.com/facebook/react-forget/pull/2825
This adds a new category of error where the compiler cannot preserve
memoization exactly how as it was originally authored. We're adding a
new category here because it's not an actionable error, and allows us to
more specifically control whether it's reportable or not.
ghstack-source-id: 9693cd42ca64b980248c6202091bdd4c827e1cd4
Pull Request resolved: https://github.com/facebook/react-forget/pull/2824
We currently support FormData for Replies mainly for Form Actions. This
supports it in the other direction too which lets you return it from an
action as the response. Mainly for parity.
We don't really recommend that you just pass the original form data back
because the action is supposed to be able to clear fields and such but
you could potentially at least use this as the format and could clear
some fields.
We could potentially optimize this with a temporary reference if the
same object was passed to a reply in case you use it as a round trip to
avoid serializing it back again. That way the action has the ability to
override it to clear fields but if it doesn't you get back the same as
you sent.
#28755 adds support for Blobs when the `enableBinaryFlight` is enabled
which allows them to be used inside FormData too.
(Unless "key" is spread onto the element.)
Historically, the JSX runtime clones the props object that is passed in.
We've done this for two reasons.
One reason is that there are certain prop names that are reserved by
React, like `key` and (before React 19) `ref`. These are not actual
props and are not observable by the target component; React uses them
internally but removes them from the props object before passing them to
userspace.
The second reason is that the classic JSX runtime, `createElement`, is
both a compiler target _and_ a public API that can be called manually.
Therefore, we can't assume that the props object that is passed into
`createElement` won't be mutated by userspace code after it is passed
in.
However, the new JSX runtime, `jsx`, is not a public API — it's solely a
compiler target, and the compiler _will_ always pass a fresh, inline
object. So the only reason to clone the props is if a reserved prop name
is used.
In React 19, `ref` is no longer a reserved prop name, and `key` will
only appear in the props object if it is spread onto the element.
(Because if `key` is statically defined, the compiler will pass it as a
separate argument to the `jsx` function.) So the only remaining reason
to clone the props object is if `key` is spread onto the element, which
is a rare case, and also triggers a warning in development.
In a future release, we will not remove a spread key from the props
object. (But we'll still warn.) We'll always pass the object straight
through.
The expected impact is much faster JSX element creation, which in many
apps is a significant slice of the overall runtime cost of rendering.
`delete` causes an object (in V8, and maybe other engines) to deopt to a
dictionary instead of a class. Instead of `assign` + `delete`, manually
iterate over all the properties, like the JSX runtime does.
To avoid copying the object twice I moved the `ref` prop removal to come
before handling default props. If we already cloned the props to remove
`ref`, then we can skip cloning again to handle default props.
We currently support Blobs when passing from Client to Server so this
adds it in the other direction for parity - when `enableFlightBinary` is
enabled.
We intentionally only support the `Blob` type to pass-through, not
subtype `File`. That's because passing additional meta data like
filename might be an accidental leak. You can still pass a `File`
through but it'll appear as a `Blob` on the other side. It's also not
possible to create a faithful File subclass in all environments without
it actually being backed by a file.
This implementation isn't great but at least it works. It creates a few
indirections. This is because we need to be able to asynchronously emit
the buffers but we have to "block" the parent object from resolving
while it's loading.
Ideally, we should be able to create the Blob on the client early and
then stream in it lazily. Because the Blob API doesn't guarantee that
the data is available synchronously. Unfortunately, the native APIs
doesn't have this. We could implement custom versions of all the data
read APIs but then the blobs still wouldn't work with native APIs. So we
just have to wait until Blob accepts a stream in the constructor.
We should be able to stream each chunk early in the protocol though even
though we can't unblock the parent until they've all loaded. I didn't do
this yet mostly because of code structure and I'm lazy.
This implements the concept of a DEV-only "owner" for Server Components.
The owner concept isn't really super useful. We barely use it anymore,
but we do have it as a concept in DevTools in a couple of cases so this
adds it for parity. However, this is mainly interesting because it could
be used to wire up future owner-based stacks.
I do this by outlining the DebugInfo for a Server Component
(ReactComponentInfo). Then I just rely on Flight deduping to refer to
that. I refer to the same thing by referential equality so that we can
associate a Server Component parent in DebugInfo with an owner.
If you suspend and replay a Server Component, we have to restore the
same owner. To do that, I did a little ugly hack and stashed it on the
thenable state object. Felt unnecessarily complicated to add a stateful
wrapper for this one dev-only case.
The owner could really be anything since it could be coming from a
different implementation. Because this is the first time we have an
owner other than Fiber, I have to fix up a bunch of places that assumes
Fiber. I mainly did the `typeof owner.tag === 'number'` to assume it's a
Fiber for now.
This also doesn't actually add it to DevTools / RN Inspector yet. I just
ignore them there for now.
Because Server Components can be async the owner isn't tracked after an
await. We need per-component AsyncLocalStorage for that. This can be
done in a follow up.
Based on:
- #28464
---
This moves the entire string ref implementation out Fiber and into the
JSX runtime. The string is converted to a callback ref during element
creation. This is a subtle change in behavior, because it will have
already been converted to a callback ref if you access element.prop.ref
or element.ref. But this is only for Meta, because string refs are
disabled entirely in open source. And if it leads to an issue in
practice, the solution is to switch to a different ref type, which Meta
is going to do regardless.
First attempt at making the linter work with advanced TypeScript syntax
Falls back to the babel parser for some advanced syntax like string template
syntax.
This is pretty hacky as it doesn't take in any parsing options that are
configured for the outer ESLint parser, not sure how that could be handled.
In prod, the `_owner` field is only used for string refs so if we have
string refs disabled, we don't need this field. In fact, that's one of
the big benefits of deprecating them.
Implements support for use:
* Teaches InferReactivePlaces to treat use() result as reactive
* Teaches FlattenScopesWithHooks to also flatten scopes with use()
Handles both `use()` and `React.use()`.
This adds rollup to the runtime and adds a new plugin to add the license banner
+ inject the `"use no memo"` directive. We need to inject it there as rollup
currently strips out unknown directives during bundling.
For now this configures rollup to strip out comments in DEV builds and
whitespace. Unfortunately there's no easy way to do this in just terser alone or
other minifiers/manglers, so I had to add prettier as well to re-format the
minified code. This does make the build a little bit slower:
``` before: yarn build 118.96s user 12.38s system 185% cpu 1:10.81 total after:
yarn build 121.55s user 12.90s system 183% cpu 1:13.17 total ```
Eventually I would like to have a similar setup to React's rollup config where
we can have DEV and prod builds. After the repo merge we could probably share or
reuse bits of React's rollup config.
This PR makes all packages share the same typescript version and updates us to
latest versions of typescript, ts-node, typescript-eslint/eslint-plugin and
typescript-eslint/parser.
I also noticed that the tsconfig we were extending (node18-strictest) was
deprecated, so I switched us over to one that's more up to date.
Also had to make a couple of small changes to the playground so that continues
to build correctly.
Previously, we would drop directives inside a component or hook but this is
problematic with reanimated which uses `'worklet'` to mark components from
compilation.
This PR adds a directive to HIRFunction and ReactiveFunction and codegens the
directive add the end. No processing is done on the directives themselves.
Babel seems to store the directives on a BlockStatement, rather than on the
Function but I've stored it on the Function types because we only support
compiling functions and the spec defines directives as occuring in the initial
statement list of a function: > A Directive Prologue is the longest sequence of
ExpressionStatements > occurring as the initial StatementListItems or
ModuleItems of a > FunctionBody, a ScriptBody, or a ModuleBody and where each >
ExpressionStatement in the sequence consists entirely of a > StringLiteral token
followed by a semicolon.
This PR was the result of a long chain of ~yak-shaving~ debugging kicked off as
a result of fixing up invariants. Where this started was that i noticed some
cases of loops where the first instance we saw of a reactive scope was after its
starting instruction. Eg instruction N would have an operand with scope
Start:End, where Start was _before_ N. One of the cases involved a phi with a
backedge. Then i noticed that we assign scopes differently for phis with and
without backedges:
```
[1] let x0 = init;
[2] if (x0 < limit) {
[3] x1 += increment;
}
x2 = phi(x0, x1);
[4] x2;
```
The phi isn't mutated _or_ reassigned after its creation, so we don't assign a
mutable range to the phi or any of its operands. We also don't create a scope
for `x`.
But change the `if` to a `while` and now the phi moves - now there's a backedge:
```
[1] let x0 = init;
[2] while (x0 < limit) {
x2 = phi(x0, x2); // now this is "mutated" later!!!
[3] x2 += increment;
}
[4] x2;
```
What was happening here is that x2 has a mutable range which is "after" the phi
instruction, so it would appear that the phi was actually being mutated later.
Ie, this was treated equivalently to the original "if" version, but with a
mutation:
```
let x = [];
if (cond) {
x = {};
}
mutate(x); // later mutation of the phi
```
But these latter two cases are different! We only need to (should) create a
mutable range for a phi _if its value is actually mutated_. If it's just being
reassigned, well then it shouldn't matter if there are back edges or not.
So this PR implements that intuition: only create a mutable range for a phi if
it is actually _mutated_ later, ie don't assign a mutable range if it is only
_reassigned_ later. Concretely in InferMutableRanges:
* InferMutableLifetimes no longer has to initialize a range for phis during the
first pass (inferMutableRangesForStores=false). We wait to see if the phi is
mutated during the main fixpoint iteration of InferMutableRanges
* The main fixpoint iteration in InferMutableRanges already aliases phi operands
if the phi is later mutated, which will extend the end of the mutable range of
all the operands accordingly.
* Finally, InferMutableLifetimes's second run
(inferMutableRangesForStores=true), we ensure that any phis mutated later have a
valid mutable range, specifically setting the `start` of the range since the
fixpoint only updates the `end` value.
A dependency D from either an instruction or scope is poisoned if there may be a
(non-linear) jump instruction between it and the start of its immediate parent
scope. Poisoned dependencies are added as conditional dependencies to their
parent scope.
(done: reduce false positives in scopes that begin after return/throw) (done:
fix bugs in recording and joining exhaustive conditional deps) (done: flesh out
commit message, clean up PR, add more fixtures)
--- \## Bug details:
Take a simple example: ```js target: { instrA; if (...) { instrB;
break target; } else { instrC; } instrD; // ... } instrE; // ... ```
This diagram shows how we represent this program in the reactive IR. - Blocks
are represented as a list of nodes. - Green nodes show instructions and value
blocks (simplified as a single instruction). - Pink nodes show terminals, which
transfer control to a subtree of nodes. <img width="450" alt="image"
src="https://github.com/facebook/react-forget/assets/34200447/930789f2-39cd-4ea8-b12a-530042807b46">
Prior to this PR, PropagateReactiveScopeDeps was incorrect because it assumed
that a block's instructions are evaluated unconditionally (which is how HIR
basic blocks work). E.g. if a reactive scope enclosed `block 1`, we assume that
`instrA` and `instrD` both will evaluate unconditionally.
This failed to account for `jump` instructions like break, continue, return, and
throw. This may result in invalid hoisting of PropertyLoads (i.e. Forget output
may throw when source does not throw). Note that other terminals (e.g. if and
loops) are not affected as they are self contained subtrees that evaluate
sequentially.
With the changes in this PR, we mark `block 1` as poisoned upon encountering the
`break` instruction. While `block 1` is active and poisoned, it will determine
how visited dependencies are added.
Here, added solid lines show unconditional dependencies, dashed lines show
conditionally accessed dependencies: - dependencies from `instrB, instrC` are
conditional because they are within conditional subtrees - dependencies from
`instrD` are conditional because it is within a poisoned block within its parent
scope.
<img width="450" alt="image"
src="https://github.com/facebook/react-forget/assets/34200447/81980f68-7e65-4bd7-ba94-3f0c26550e5c">
--- Recapping an offline discussion with @josephsavona: this pass would really
benefit from operating on HIR. The minimal work needed for this pass to run on
HIR is to rewrite and reorder `AlignReactiveScopesToBlockScopes` to operate on
HIR.
The following diagram shows what HIR blocks look like for the same code.
Evaluating hoistable PropertyLoad dependencies for a scope enclosing
`instr{A-D}` is much simpler: just evaluate whether the PropertyLoad evaluates
for every path between `bb0` and `bb4`. <img width="250" alt="image"
src="https://github.com/facebook/react-forget/assets/34200447/44b38939-defb-4b29-878d-4445ec6ccc06">
---
conditionals
This change is needed for #2752. To minimize renaming `error.fixture` ->
`fixture` files, I'm reordering this PR to earlier in the stack.
Prior to the fix in #2752, we only expected unconditional accesses within
`depsInCurrentConditional`, which records instructions directly within a
conditional block (not including nested conditional blocks).
RFC: we can either retain break/continue target ids (instead of pruning them in
`buildReactiveFunction`) or re-implement the same logic in `propagateScopeDeps`
(ignore implicit breaks; match unlabeled break / continues to their closest loop
/ while parent terminal).
If we go ahead with this approach, I'll clean up this PR (add relevant types and
comments)
The reanimated babel plugin specifically looks for args to their hooks that are
callbacks, then it workletizes the body of that callback so it can run on the
main thread.
But, forget extracts that callback into a temporary variable and then replaces
the previously inlined callback as an identifier, so that breaks reanimated's
babel plugin. so what happens is some of the previously workletized functions no
longer do after forget runs, which throws a runtime error about a non-worklet
function running on the main thread.
Reanimated expects this: ``` const animatedGProps = useAnimatedProp(function ()
{ ... }) ```
But forget does this: ``` const t0 =function () { ... } const animatedGProps =
useAnimatedProp(t0) ```
With the type definitions, Forget no longer assumes the args to reanimated APIs
escape so Forget does not memoize and they stay as is.
Our current validation fails to detect some invalid cases of mutable ranges —
namely, ranges that are fully or partially uninitialized, with start or
start+end still set to zero.
This PR fixes these cases, starting by validating that the ranges for _all_
reactive scopes are valid: start >= 1, and end <= (last instr id + 1). This
exposed the invalid cases, which are also fixed here:
* During AnalyzeFunctions, we need to reset identifier ranges and scopes when
exiting an inner function. This has to happen *after* the effects have been
translated to the function deps/context operands, in order for
InferRefenceEffects to continue working on the outer function. Previously I did
this at the start of InferReactiveScopeVariables, but that's insufficient bc the
incorrect ranges could also influence InferMutableRanges. AnalyzeFunctions is
the point at which we compute the ranges for identifiers in the inner function,
so it's the most ideal place to clean those up so they don't influence the outer
function.
* In InferMutableLifetimes, we need to ensure that context variable identifiers
end up with a mutable range starting where they are declared, and ending with
their last assignment. We now track declarations and extend their mutable range
to account for each reassignment.
Fixes the repro added in 947832009997bf9149e88e583c46cc39f6a6136c - previously
when computing mutable ranges of phis, we didn't check that all operands had
been visited. This meant that a backedge could allow a phi's mutable range to
start at 0. Then in PropagateScopeDeps, we might see reject dependencies of a
scope since they appeared to start after a scope — only because the scope's
start was incorrectly too early.
The fix here is to initially set phi.id mutable ranges based on only on operands
that are already visited. Then, during/after the fixpoint iteration of
InferMutableRanges, we start account for all operands since we know they've been
visited at least once and have a real range.
Updates InferReactiveScopeVariables to first prune scopes attached during
AnalyzeFunctions. This ensures that after this pass the only scopes that exist
on identifiers in the outer program are those that the pass explicitly inferred,
and not accidentally leftover.
We share identifiers between outer functions and inner function expressions,
which means that scopes inferred during AnalyzeFunctions can be retained on
Identifiers in the outer function. If InferReactiveScopeVariables doesn't happen
to visit an identifier we currently retain that scope information and use it
when constructing scopes, even if there doesn't technically need to be a scope
for that value.
Fixed in the next PR.
What happens here is that the phi node for `i` has its mutable range set to
start at 0, because it has a back edge and we haven't initialized the mutable
ranges of all its operands yet when we iterate the operands and set range.start
= min(start of operand starts).
Then the corresponding scope has its range set to start at 0 too. When
PropagateScopeDeps runs it sees that `b` is from instruction 1, which is after
the start of the scope (0), so it thinks `b` isn't a valid dependency.
The fix is in InferMutableRanges, where we need to make sure that phis ignore
their operand's ranges until those ranges are initialized.
Correct eslint-plugin-react-compiler dependencies
- The eslint plugin doesn't actually depend on the babel plugin as it compiles
in the dependencies. - `zod` and `zod-validation-error` were missing, but
required to run the plugin. - Update to the `hermes-parser` dependency just to
keep it updated.
Our logic to detect hoisting relies on Babel's `isReferencedIdentifier()` to
determine whether a reference to an identifier is a reference or a declaration.
The idea is that we want to find references to variables that may be hoistable,
before the declaration — the definition of hoisting. But due to the bug in
isReferencedIdentifier, we skipped over reassignments of hoisted variables. The
hack here checks if an identifier is a direct child of an AssignmentExpression,
ensuring we visit reassignments.
Adds examples for closures that reassign hoisted let bindings. We currently
compile these as context variables without hoisting, so we hit an invariant in
InferReferenceEffects when the variable is referenced before being declared.
`let` bindings of context variables are lowered to a DeclareContext +
StoreContext, which breaks codegen for `for` loops which expect that all
statements of the init block will lower to variable declarations. The two
instructions produce a variable declaration and a reassignment.
If we have a switch with only a default case, then that code will be executed
unconditionally. PropagateScopeDeps can take advantage of this to record
dependencies in these cases as unconditional, which avoids the issue seen in the
previous PR.
---
Thanks to @josephsavona for finding this bug. This is another example of why we
really want hir-everywhere.
Forget output currently nullthrows because we believe `obj.a` is run
unconditionally in source (missing the break/returns out of this scope)
I haven't debugged to understand exactly why this pattern fails, but there are a
few instances of this internally. It's especially weird because
```javascript
// @enableAssumeHooksFollowRulesOfReact
@enableTransitivelyFreezeFunctionExpressions
function Component(props) {
const [_state, setState] = useState();
const a = () => {
return b();
};
const b = () => {
return (
<>
<div onClick={() => onClick(true)} />
<div onClick={() => onClick(false)} /> // <---- only repros if there's a second
call!
</>
);
};
const onClick = (value) => {
setState(value);
};
return <div>{a()}</div>;
}
```
Here, if `b()` only had one nested function expression that called `onClick` it
would work. Also, if we disable `@enableTransitivelyFreezeFunctionExpressions`
then it works.
But the combination of multiple calls plus that mode causes "context variables
are always mutable". I'm guessing we're freezing `onClick` twice and the second
time reports an error since it calls `setState`.
We don't have a `DestructureContext` equivalent of `StoreContext`, so variables
that are declared via destructuring and later reassigned trigger the invariant
that all mentions of a variable must be consistently local or context. The next
PR adds a todo for this case.
We inadvertently think the type annotation on the function expression param is
an identifier and create a LoadLocal for it, which fails. This happens to trip
up on the InferReferenceEffects initialization check, which we had assumed would
only fire for invalid hoisting cases (hence the specific error message).
When PruneMaybeThrows removes maybe-throw terminals, it's possible that the
block in question reassigned a value s.t. it appears as a later phi operand.
That phi has to be rewritten to reflect the updated predecessor block.
Here we track these rewrites (transitively) and rewrite phi operands
accordingly.
Found when running the compiler on a large swath of internal code.
PruneMaybeThrows rewrites terminals, but the logic to update subsequent phis was
incorrectly dropping phis rather than rewriting them. Fixed in the next PR.
---
Reusing optionalMemberExpression nodes recently led to a bug when compiling
Forget playground.
```js
// the two a?.b's here should be different nodes!
if (a?.b !== $[0]) {
// ...
$[0] = a?.b;
}
```
Forget playground uses `babel-plugin-react-forget` and `next/babel`. Reusing the
same node in two positions in the AST lead to invalid mutations:
- the first `a?.b` is visited and transpiled to `a === void 0 ? ...`, which (1)
inserts nodes between the original node and its parent and (2) mutates `a?.b` in
place to a non-optional call
- the second `a?.b` in source gets updated to `a.b` and does not get visited
again
```js
// Source in `EditorImpl.tsx`
compilerOutput.kind === "err" ? compilerOutput.error.details : []
// Forget transformed:
if ($[2] !== compilerOutput.kind || $[3] !== compilerOutput.error?.details) {
t4 = compilerOutput.kind === "err" ? compilerOutput.error.details : [];
$[2] = compilerOutput.kind;
// this is good!
$[3] = compilerOutput.error?.details;
$[4] = t4;
} else {
t4 = $[4];
}
// After next/babel
if ($[2] !== compilerOutput.kind || $[3] !== ((_compilerOutput$error =
compilerOutput.error) === null || _compilerOutput$error === void 0 ? void 0 :
_compilerOutput$error.details)) {
t4 = compilerOutput.kind === "err" ? compilerOutput.error.details : [];
$[2] = compilerOutput.kind;
// Oh no!!
$[3] = _compilerOutput$error.details;
$[4] = t4;
} else {
t4 = $[4];
}
```
---
This should make it easier to grep through error diagnostics to understand state
of the codebase:
- no matching dependences -> likely that source is ignoring eslint failures
- differences in ref.current access -> non-backwards compatible refs
- subpath -> should be fixable in the compiler (unless source is ignore eslint
failures)
---
Previously (in #2663), we check that inferred dependencies exactly match source
dependencies. As @validatePreserveExistingMemoizationGuarantees checks that
Forget output does not invalidate *any more* than source, we now allow more
specific inferred dependencies when neither the inferred dep or matching source
dependency reads into a ref (based on `.current` access).
See added comments for more details
I need to do more debugging to figure out exactly why the example earlier fails
— but whatever it is, it's clearly a matter of the fbt plugin relying on some
specifics of source locations.
Here we just detect multiple instances of `<fbt:enum>` within a given `<fbt>`
tag and throw a todo.
<img width="553" alt="Screenshot 2024-03-19 at 4 41 15 PM"
src="https://github.com/facebook/react-forget/assets/6425824/e87ee704-6c67-4e10-824b-71e97e7e19f5">
Slightly improves source locations for JSX elements so that the opening and
closing tag have distinct locations that match up with source. The identifier
itself within the closing tag still has the wrong location, but at least this is
an improvement.
Doesn't fix the fbt thing but it was worth a try.
Fbt enums appear to rely on source locations and something that we're doing
(maybe destructuring?) isn't preserving locations such that the fbt plugin
breaks.
Fbt violates the JSX spec by using a lowercase function as a tagname, even
though lowercase names are reserved for builtins. Here we detect cases where
there is an `<fbt>` tag where `fbt` is a local identifier and throw a todo.
The example earlier in the stack had unreachable code in the output because
there was an unnecessary memoization block around an assignment. This was a
holdover from before we moved the logic to expand mutable ranges for phis from
LeaveSSA to InferMutableRanges. We were conservatively assigning a mutable range
to all variables with a phi, even those that didn't strictly need one.
Removing the range extension logic in LeaveSSA fixed the issue, but uncovered
the fact that AlignReactiveScopesToBlockScopes was missing a case to handle
optionals.
## Test Plan
Synced internally and ran a snapshot/comparison of compilation before/after
(P1197734337 for those curious). The majority of components get fewer memo slots
thanks to not needing to memoize non-allocating value block expressions like
ternaries/optionals. In a few cases, the fact that we're no longer assigning a
mutable range for value blocks (unless there is actually a mutation!) means we
get more fine-grained memoization and increase the number of memoization blocks.
So overall this appears to be correct, improve memoization, and reduce code
size.
Extracts a helper from the repro earlier in the stack into a helper in
shared-runtime. This makes it easy to verify that memoization is actually
working.
This case is specific to early return inside an inlined IIFE (which can often
occur as a result of dropping manual memoization). When we inline IIFEs, as a
reminder we wrap the body in a labeled block and convert returns to assignment
of a temporary + break out of the label.
Those reassignments themselves are getting a reactive scope assigned since the
reassigned value has a mutable range. They don't really need a mutable range or
scope, though. And then the presence of the `break` statements means that we can
sometimes exit out of the scope before reaching the end - leading to unreachable
code.
This can only occur though where _all the values are already memoized_. So the
code works just fine and even memoizes just fine - it's just that we have some
extraneous scopes and there is technically unreachable code. I'll fix in a
follow-up, adding a repro here.
of dependencies from source
---
`validatePreserveExistingMemoizationGuarantees` previously checked
- manual memoization dependencies and declarations (the returned value) do not
"lose" memoization due to inferred mutations
```
function useFoo() {
const y = {};
// bail out because we infer that y cannot be a dependency of x as its
mutableRange
// extends beyond
const x = useMemo(() => maybeMutate(y), [y]);
// similarly, bail out if we find that x or y are mutated here
return x;
}
```
- manual memoization deps and decls do not get deopted due to hook calls
```
function useBar() {
const x = getArray();
useHook();
mutate(x);
return useCallback(() => [x], [x]);
}
```
This PR updates `validatePreserveExistingMemoizationGuarantees` with the
following correctness conditions:
*major change* All inferred dependencies of reactive scopes between
`StartMemoize` and `StopMemoize` instructions (e.g. scopes containing manual
memoization code) must either:
1. be produced from earlier within the same manual memoization block
2. exactly match an element of depslist from source
This assumes that the source codebase mostly follows the `exhaustive-deps` lint
rule, which ensures that deps lists are (1) simple expressions composing of
reads from named identifiers + property loads and (2) exactly match deps usages
in the useMemo/useCallback itself.
---
Validated that this does not change source by running internally on ~50k files
(no validation on `main`, no validation on this PR, and validation on this PR).
---
Previously, we always emitted `Memoize dep` instructions after the function
expression literal and depslist instructions
```js
// source
useManualMemo(() => {...}, [arg])
// lowered
$0 = FunctionExpression(...)
$1 = LoadLocal (arg)
$2 = ArrayExpression [$1]
$3 = Memoize (arg)
$4 = Call / LoadLocal
$5 = Memoize $4
```
Now, we insert `Memoize dep` before the corresponding function expression
literal:
```js
// lowered
$0 = StartMemoize (arg) <---- this moved up!
$1 = FunctionExpression(...)
$2 = LoadLocal (arg)
$3 = ArrayExpression [$2]
$4 = Call / LoadLocal
$5 = FinishMemoize $4
```
Design considerations:
- #2663 needs to understand which lowered instructions belong to a manual
memoization block, so we need to emit `StartMemoize` instructions before the
`useMemo/useCallback` function argument, which contains relevant memoized
instructions
- we choose to insert StartMemoize instructions to (1) avoid unsafe instruction
reordering of source and (2) to ensure that Forget output does not change when
enabling validation
This PR only renames `Memoize` -> `Start/FinishMemoize` and hoists
`StartMemoize` as described. The latter may help with stricter validation for
`useCallback`s, although testing is left to the next PR.
#2663 contains all validation changes
Remove private header from playground
Before we miss removing this from the public release, I think we can remove this
header now already. We're still behind a secret URL + password.
Fixes T180504437. We expected `<fbt:param>` to always have no surrounding
whitespace or have both leading and trailing whitespace, it can have one but not
the other, though such cases are rare in practice.
Repro from T180504728 which reproduced internally and on playground, neither of
which have #2687 yet. That PR (earlier in this stack) already fixes the issue,
so i'm just adding the repro to help prevent regressions.
While i'm here, we know that there are a variety of cases that are not supported
yet around combining value blocks with other syntax constructs. Since we're
aware of these cases and detect them, we can make this a todo instead of an
invariant.
We need to revisit the conversion from value blocks into ReactiveFunction. Or
just revisit ReactiveFunction altogether (see my post about what this would look
like). For now, makes this case a todo.
"Support" in the sense of dropping these on the floor and compiling, rather than
bailing out with a todo.
We already don't make any guarantees about which type annotations we'll preserve
through to the output, so it seems fine for now to just drop type aliases.
I addressed some of the cases that lead to this invariant but there were still
more. In this case, we have scopes like this:
```
scope @1 declarations=[t$0] {
let t$0 = ArrayExpression []
if (...) {
return null;
}
}
scope @2 deps=[t$0] declarations=[t$1] {
let t$1 = Jsx children=[t$0] ...
}
```
Because scope 1 has an early return, PropagateEarlyReturns wraps its contents in
a label and converts the returns to breaks:
```
scope @1 declarations=[t$0] earlyReturn={t$2} {
let t$2
bb0: {
let t$0 = ArrayExpression []
if (...) {
t$2 = null;
break bb0;
}
}
}
scope @2 deps=[t$0] declarations=[t$1] {
let t$1 = Jsx children=[t$0] ...
}
```
But then MergeReactiveScopesThatInvalidateTogether smushes them together:
```
scope @1 declarations=[t$1] earlyReturn={t$2} {
let t$2
bb0: {
let t$0 = ArrayExpression [] // <--- Oops! We're inside a block now
if (...) {
t$2 = null;
break bb0;
}
}
let t$1 = Jsx children=[t$0] ...
}
```
Note that the `t$0` binding is now created inside the labeled block, so it's no
longer accessible to the Jsx instruction which follows the labeled block. This
isn't an issue with promoting temporaries or propagating outputs, but a simple
issue of the labeled block (used for early return) introducing a new block
scope. The solution here is to simply reorder the passes so that we transform
for early returns after other optimizations. This means the jsx element will
basically move inside the labeled block, solving the scoping issue:
```
scope @1 declarations=[t$1] earlyReturn={t$2} {
let t$2
bb0: {
let t$0 = ArrayExpression [] // ok, same block scope as its use
if (...) {
t$2 = null;
break bb0;
}
let t$1 = Jsx children=[t$0] // note this moved inside the labeled block
}
}
```
I addressed some of the cases that lead to this invariant but there were still
more. In this case, we have scopes like this:
```
scope @1 declarations=[t$0] {
let t$0 = ArrayExpression []
if (...) {
return null;
}
}
scope @2 deps=[t$0] declarations=[t$1] {
let t$1 = Jsx children=[t$0] ...
}
```
Because scope 1 has an early return, PropagateEarlyReturns wraps its contents in
a label and converts the returns to breaks:
```
scope @1 declarations=[t$0] earlyReturn={t$2} {
let t$2
bb0: {
let t$0 = ArrayExpression []
if (...) {
t$2 = null;
break bb0;
}
}
}
scope @2 deps=[t$0] declarations=[t$1] {
let t$1 = Jsx children=[t$0] ...
}
```
But then MergeReactiveScopesThatInvalidateTogether smushes them together:
```
scope @1 declarations=[t$1] earlyReturn={t$2} {
let t$2
bb0: {
let t$0 = ArrayExpression [] // <--- Oops! We're inside a block now
if (...) {
t$2 = null;
break bb0;
}
}
let t$1 = Jsx children=[t$0] ...
}
```
Note that the `t$0` binding is now created inside the labeled block, so it's no
longer accessible to the Jsx instruction which follows the labeled block. This
isn't an issue with promoting temporaries or propagating outputs, but a simple
issue of the labeled block (used for early return) introducing a new block
scope. The solution (in the next PR) is to simply reorder the passes so that we
transform for early returns after other optimizations. This means the jsx
element will basically move inside the labeled block, solving the scoping issue:
```
scope @1 declarations=[t$1] earlyReturn={t$2} {
let t$2
bb0: {
let t$0 = ArrayExpression [] // ok, same block scope as its use
if (...) {
t$2 = null;
break bb0;
}
let t$1 = Jsx children=[t$0] // note this moved inside the labeled block
}
}
```
This was an oversight in codegen. The entire pipeline supports multiple values
in a for initializer, but codegen was dropping all but the first initializer.
Fixes T180504437. In MergeOverlappingReactiveScopes we track the active scopes
and mark them as "ended" when reaching the first instruction after their mutable
range. However, in cases of interleaving that will be merged, we could
previously mark a scope as complete when it's original range was completed, even
though the range would get extended post-merge. The fix here detects
interleaving earlier, and eagerly updates the mutable ranges of the merged
scopes to ensure that neither is "ended" earlier than it should.
The repro here fails without this change.
Fixes T175282980. InferReactiveScopeVariables had logic to force assigning a
scope to MethodCall property lookups with the idea of forcing the method call
lookup to be in the same scope as the method call itself. But this doesn't work
if we never assign a scope to the method call! That can happen if we're able to
infer that the method call produces a primitive and doesn't need memoization.
This PR changes things so that:
* InferReactiveScopeVariables no longer assumes that MethodCall property values
need a scope
* We run a separate pass that ensures that _if_ a MethodCall has a scope, that
it's property is in the scope, and that otherwise its property doesn't get a
scope. This is similar to the existing passes that force a single scope for
related instructions like ObjectMethod+ObjectExpression and fbt operands/calls.
Fixes T180509722. What happened is that the logic in LeaveSSA to find
declarations within for initializers wasn't working with try/catch because the
initializer block gets broken up with a maybe-throw after every instruction that
can throw. These maybe-throws can then get turned into gotos by
PruneMaybeThrows, so LeaveSSA has to handle both cases.
The new logic scans from the start of the init block until reaching the end, and
creates declarations for all StoreLocals. Note that we don't yet support
maybe-throw in value blocks — that's already a todo — so the change here simply
allows us to compile farther until reaching that other todo. But i've
double-checked the HIR and it looks correct for this case, so it should just
work once we fix that todo. I've also added a comment to help us remember (and
of course, we'd have to add a snap fixture too)
---
(This came out of running a sync and observing hundreds of bailouts due from
this validation)
Reading `fnType` from environment overgeneralizes, as inner functions are
usually not the type of the outer react function.
```
// Component type
function Component() {
// not Component type
const helper = () => {...};
}
```
Let's attach fnType to `HIRFunction` and use that for our inference +
validations
Fixture from T175283039, a reassignment within an expression can sometimes
generate a StoreLocal within a value block. Depending on the case this can end
up as the last instruction of the block, which then hits an invariant.
Adds a todo in HIRBuilder, before we prune unreachable code we check if there
were any function expressions. Realistically that's only going to occur for
hoisted functions, so this lets us target a todo rather than hit an invariant.
For T175282529. We already detect hoisted functions and have a specific todo for
them, but in this case the function is in unreachable code that gets pruned
during BuildHIR. The later check for hoisted functions doesn't find it.
One of our visitors wasn't visiting TryTerminal's handlerBinding, which meant
that we missed renaming those identifiers in RenameVariables. I also updated the
printers to print this binding.
Within a try/catch, every instruction is followed by a maybe-throw terminal.
This currently breaks the logic in BuildReactiveFunction which tries to
reassemble the value block, since it isn't expecting the maybe-throw.
Conceptually the logic should just ignore it — we could even flatten away
maybe-throw terminals before this pass — but for now since this pattern is rare
we can just make it a todo.
For T181507827 — adds an invariant in codegen when emitting identifiers to
ensure that we only create babel Identifier nodes for nodes that the compiler
has explicitly promoted to valid, named identifiers. This means that we'll fail
for unnamed temporaries (previously caught), as well as promoted temporaries
that somehow didn't get renamed by RenameVariables (newly caught).
Adds a visitor to collect all the globals that are referenced within the
function, and then uses this list to avoid synthesizing variables with
conflicting names. This is used in both RenameVariables (for promoted
temporaries) and Codegen (for `$` and change variables only, so far, but this
can be extended in follow-ups).
This is a key part of avoiding generating conflicting names in our output. To
start, RenameVariables now returns a Set of the unique identifier names that
exist in the function. Codegen uses this to avoid generating duplicate names for
change variables and for the `$` useMemoCache variable. Rather than always emit
`$` or `c_N`, codegen checks that this name would not conflict and appends an
incrementing suffix until it finds a unique name.
Note that it's still possible for us to generate conflicts with global
variables, both during RenameVariable and Codegen. The next step will be to
avoid conflicts with globals.
Another title for this PR could be "Yet another reason for HIR-everywhere"
ReactiveFunctionVisitor doesn't traverse into HIRFunctions from
FunctionExpression and ObjectMethod values. This means that
PromoteUsedTemporaries and RenameVariables also weren't traversing into such
functions, and those values weren't getting promoted and renamed correctly.
This PR updates ReactiveFunctionVisitor with a method that can optionally be
invoked to traverse an HIRFunction and call the appropriate visitor methods.
PromoteUsedTemporaries and RenameVariables invoke this to ensure they visit all
places, even in nested HIRFunctions.
I realized that codegen still had a fallback for generating identifier nodes for
unnamed temporaries. This PR updates codegen to throw if it needs to generate an
identifier for a temporary, and updates earlier passes to promote temporaries to
named values in all the cases that were missed:
* BuildHIR needs to promote temporaries for temporaries in destructuring
bindings and catch clause bindings
* PromoteUsedTemporaries has to promote temporaries for destructured function
parameters or function params that are context variables.
Uses an enum for Identifier.name to distinguish originally named identifiers vs
promoted temporaries. An opaque type for the named identifier variant makes it
hard to accidentally create that type.
When the compiler promotes temporary values to named variables, we currently
eagerly assign a name using the temporary's IdentifierId. This means that we're
sort of stuck with this name later in compilation, and RenameVariables can't be
100% sure whether a 't0' variable is a temporary or not. As a result, the names
of these promoted temporaries is influenced by how many temporaries we happened
to create during compilation (and what the next available identifier id was),
making them fluctuate more as we iterate on the compiler.
This is an RFC for showing how we can stabilize these names. The key elements:
* Distinguish promoted temporaries from other named identifiers. Here we use a
hack, naming them starting with '#t' or '#T', since '#' isn't a valid identifier
starting point. This lets us keep all of our logic that looks for non-null
identifiers names to distinguish named/unnamed, while also distinguishing real
names from generated names (if this was Rust, we'd use an Enum and have a
"isNamed()" method on it that was true for real/temporary names and false
otherwise)
* In RenameVariables, detect generated names and fall back to generating the
next available `tN`-style name (or `TN` for JSX tags).
* To reduce thrash overall, RenameVariables no longer keeps a global "next id"
value that uses to distinguish all conflicting identifiers, instead we restart
at 0 whenever we find a conflict, and keep bumping until we find a free name.
Thus if both `foo` and `bar` had conflicts, we previously would end up with
`foo$0` and `bar$1` as the deduped names, but now will end up with `foo$0` and
`bar$0`.
## RFC
I'm open to feedback on the approach. Two main questions:
* How to annotate promoted temporaries. The most type-safe option is to change
`Identifier.name` to be a union of `{kind: 'named', value: string} | `{kind:
'promoted', value: string} | `{kind: 'temporary'}` though TS then wouldn't allow
`identifier.name.value` (even as nullable) since it doesn't exist on one of the
variants. Maybe we could type the temporary one as `{kind: 'temporary', value?:
null}` so the value has to be null but you can always access that property?
* ?? Other concerns about the approach? We could keep the global
auto-incrementing id rather than attempting to reset to 0 for each conflict.
The previous implementation used IdentifierId, but since this pass operates
after LeaveSSA the identifier ids are no longer distinct for different SSA
instances. Instead we use the Identifier instance, which preserves SSA
information (even ever LeaveSSA) and allows distinguishing between variables
whose value always changes vs variables that may be reassigned such that they
don't always invalidate.
In the future when we use HIR everywhere, this pass should use the HIR CFG to
understand that phi nodes whose operands all will always invalidate can also be
treated as always invalidating.
## Test Plan
Synced to www, 91 files have output changes
(https://fburl.com/everpaste/3e3hjpjs). I spot checked these and confirmed that
they are all from cases where there was already missing memoization of earlier
values, where we now can prune later reactive scopes that depend on the
un-memoized values.
Implements the optimization described in the previous PR: if we know that a
scope's dependency will _always_ invalidate (it is not memoized and it is
guaranteed to be a new object if the instruction executes, such as an array or
object literal), then we can prune that scope. The invalidation is transitive:
we track always-invalidating types from within scopes, and if their scope gets
invalidated we prune downstream scope that depend on them.
## Test Plan
Tested via #2639 - see https://fburl.com/everpaste/3e3hjpjs. 91 files change
output due to reactive scopes which would always invalidate due to always
invalidating dependencies.
These fixtures demonstrate how currently, even if the dependency of a scope
doesn't get memoized (ie the scope gets pruned), we don't remove later scopes
that depend on that value. Those later scopes will always invalidate, so we
might as well remove them.
Summary: Currently Forget bails on mutations to globals within any callback function. However, callbacks passed to useEffect should not bail and are not subject to the rules of react in the same way.
We allow this by instead of immediately raising errors when we see illegal writes, storing the error as part of the function. When the function is called, or passed to a position that could call it during rendering, we bail as before; but if it's passed to `useEffect`, we don't raise the errors.
Infer if a function is a component or hook when we're deciding to compile a
function and store that in the environment.
This is used in passes like InferReferenceEffects rather than having to re-parse
the name in each pass.
Previously, Forget would throw if _any_ of the arguments to a component are
modified. This isn't quite right as a ref argument can be modified.
This PR assumes the second argument of a component to be a ref and allows it to
be mutable.
A future PR will add types to this argument so the validateRefAccessDuringRender
can catch if ref is mutated in render. This PR contains a todo test for this.
Rather than force scopes to be created for primitives within
InferReactiveScopeVariables, here we move the creation of scopes for these
instructions to a later pass. Later in the pipeline we have more context, such
as whether e.g. a primitive or propertyload is being accessed within a scope or
not, and whether it therefore needs its own scope or not.
Currently we allocate all reactive scopes during a single pass,
InferReactiveScopeVariables, using a local incrementing number to assign
ScopeIds. This means we can't easily create additional scopes later since we
don't know the next available scope id.
Here we add `Environment.nextScopeId` and use that to synthesize scope ids.
Looking up certain properties on a hook is a common pattern for logging.
It's non-ideal but it's not a bug to do this.
This updates Forget to not error on this pattern.
filepath
Internal rollout currently has a good number of test failures.
`enableEmitInstrumentForget` can help developers understand which functions /
files they should look at:
```
// input
function Foo() {
userCode();
// ...
}
// output
function Foo() {
if (__DEV__ && inE2eTestMode) {
logRender("Foo", "/path/to/filename.js");
}
const $ = useMemoCache(...);
userCode();
}
```
The code for value block handling assumes a small set of terminal kinds, but
try/catch causes the entire body to get wrapped in MaybeThrow terminals. We need
to skip over these and delegate to the inner content.
This invariant interpolated values into the `reason` which prevent our internal
tooling from grouping related errors. This PR updates to make the reason static
and interpolate the description.
Fixes T173101142 — we previously computed incorrect function expression
dependencies for JSXMemberExpressions. This PR applies similar logic to
JSXMemberExpression as we use for MemberExpression.
## Test Plan
Synced internally, only one file changes output. I manually investigated to
confirm — the change is that a function expression's dependencies are more
precise and correct. See https://fburl.com/everpaste/4dqewxqv
---
No changes to snap or sprout's functionality.
Tweaks to consolidate sprout into snap while keeping its simple interface and
most developer patterns.
- to keep `filter` mode fast, we do not run sprout in filter mode
- sprout is run in non-filter mode for both test and update
~~Small qol improvement: `--watch` will start you in `filter` mode~~
### Cost of this change
`performance.now()` is quite noisy due to background processes and ThreadPool
logic (especially with asymmetric task distribution), so I used
`process.cpuUsage` which reports time spent in user-space. This was much less
noisy (1-4% standard dev / mean)
Running all tests becomes slower by ~50%. Initial runs are slower because they
load in Forget's `require` chains.
- 23.9s previous initial run
- 34.6s current initial run
- 11.5s previous subsequent runs
- 15.4s current subsequent runs
Running filtered tests remains very fast (~100ms on the average case)
---
Additional modes or commands could be added as needed (e.g. run tests in filter
mode, with sprout output)
Adds some test cases for hook calls in object methods. Initially we didn't catch
these because InferTypes doesn't actually visit ObjectMethod bodies. Once we fix
that we correctly reject these examples.
> Don’t call Hooks inside loops, conditions, or nested functions
Per https://react.dev/warnings/invalid-hook-call-warning#breaking-rules-of-hooks
it is invalid to call hooks inside function expressions. We now validate this by
default, i'll verify internally before landing.
Note the validation is somewhat more conservative and we only disallow known
hook calls here, this seems like a reasonable tradeoff but i'm open to
suggestions. We could reuse the same known/potential hook mechanism here but it
would take some more refactoring.
Updates the compiler to understand Flow hook syntax. Like component syntax, in
infer mode hooks are compiled by default unless opted out.
Looking ahead, i can imagine splitting up our compilation modes as follows:
* Annotations: opt-in explicitly
* Declarations: annotations + component/hook declarations
* Infer: annotations, component/hook declarations, + component/hook-like
functions
This also suggest an alternative annotation strategy: "use react" (or "use
component" / "use hook") as a general way to tell the compiler that a function
is intended for React. Then opting out of memoization could do "use
react(nomemo)".
Updates LowerReactiveScopes to rewrite to a ReactiveFunctionValue
(ReactiveFunction-based) instead of a FunctionExpression (HIR-based). This lets
us include terminals and even nested reactive scopes in the result.
Per the previous PR, we don't have a way to rewrite an arbitrary subset of a
ReactiveFunction into a function expression, since FunctionExpression's contents
is still in HIR.
While long-term our plan is to move to HIR everywhere, this PR adds a stopgap of
adding a ReactiveFunctionValue variant of ReactiveValue. As a reminder,
ReactiveValue is a union of (HIR) InstructionValue | SequenceExpression |
LogicalExpression | ConditionalExpression.
For now i did a first stab at the visitors and transforms with the idea that:
* By default, visitors/transforms _don't_ look into these function expressions,
since we didn't previously traverse into (HIR-based) FunctionExpression either
* But there is a visitor/transform method that you can override if you need to.
Adds an example demonstrating why we need the ability to rewrite parts of a
ReactiveFunction into a function expression. Here, the reactive scope needs to
contain an `if` terminal, but we can't put a ReactiveIfTerminal inside a
function expression, since that expects HIR.
There are two main paths forward:
* Use HIR everywhere. I wrote this up and we're all agreed, it's just a bunch of
work.
* Add an alternative FunctionExpression variant to ReactiveFunction
For now i'm going to take the second route.
We want to start moving away from "Forget", so this PR adds support "use memo"
and "use no memo"
I've left "use forget" and "use no forget" directives unchanged for now, as we
need to migrate existing users first and then come back and delete support for
these directives.
Add code frame to snap errors
This should make it easier (possible) to see if errors point at the right lines.
No idea why I had to add 1 to the column, you'd think it's all babel-standard
(whatever it is) and there wouldn't be off by one errors, but I'm not quite in
the mood to debug babel issues more then necessary right now...
This caused a build error when Forget was used in an Expo app as the
react-forget-runtime package was itself being compiled with Forget. This broke
Metro as metro serializes modules to iifes, but the import syntax that was
injected by the useMemoCachePolyfill flag was left behind
In practice I don't think the runtime package needs to ever be compiled by
Forget, so this PR opts out the whole file. This would also prevent builds from
breaking if someone decided to use the "all" compilation mode.
Test plan: Ran the expo app and verified that it now builds with no errors
Currently we only allow adding the directive to function bodies, but there may
be cases where we want to always opt out an entire module from being compiled by
Forget
A labeled block will generally end with an implicit break out of the label.
However, if there are no _explicit_ breaks to the label, we'll end up with a
ReactiveFunction along the lines of:
```
bb1: {
...instructions with no explicit `break bb1`...
(implicit) break;
}
```
The `PruneUnusedLabels` pass removes such unused labels, inlining the content of
label terminal into the surrounding block. However, we weren't pruning the
`break`! This wasn't a problem in practice since codegen, and future passes,
would just ignore this. But it's more correct to go and find these unnecessary
implicit breaks and prune them, which this PR does.
Again, this shouldn't have any impact other than producing cleaner
ReactiveFunction data during debugging.
Continuing on my quest to clean up our feature flags, the logic for merging
consecutive feature flags is stable. Let's remove
`@enableMergeConsecutiveScopes` since this is enabled everywhere.
Some components stop being components over time and are used as regular
functions instead, but they may have lingering hook calls. Those hook calls make
it so the capitalized function calling them do not error (they appear to be a
function to existing eslint rules), but they are nonetheless unsafe to memoize.
This diff adds a conservative option to bail out on all capitalized function
calls.
There are a handful of known-non-component capitalized functions, like
`Boolean`, `String`, and `Number`. This diff also adds the ability to supply
capitalized function names that should not be considered in this analysis.
I added three tests:
1. Ensure an error occurs in the obvious case
2. Ensure an error occurs when the value is aliased simply
3. Ensure the allowlist works
This is my first commit so please go hard on me. I was unsure about where this
code should live, so please nitpick.
The hook guards are incompatible with using a forget-runtime. Specifically,
forget-runtime needs to make a call to `useState()` or some other hook to attach
data to the fiber, but all the builtin hooks are overridden to disallow calling
them outside of explicit boundaries. We'd either have to wrap the useMemoCache
call in a push/pop to allow it to call other hooks, or as in this PR, just move
it outside the enforcement.
These validations needs to be able to transitively check for violations within
function expressions, without immediately erroring. So the inner "-Impl" helpers
return a Result. But the outer, exported validate functions don't need to return
a Result, especially since TS has no Rust-style enforcement that return values
are actually used. Unwrapping within the validation means the caller can't
forget to do so and inadvertently silence the errors.
I had split this up from the main validation since function validation was less
precise; now that previous PRs fix the false positives we can remove this extra
flag.
This pass doesn't really make sense in light of
`@enableTransitivelyFreezeFunctionExpressions`. The original idea of
ValidateFrozenLambdas was that trying to pass a "mutable" lambda to a frozen
value was invalid. But since then we've realized that the better heuristic is
that freezing a lambda is transitive.
Rewrites the validation to not rely on the mutable range of functions to
determine whether they are called or not, since the range can be extended for
other reasons (they happen to reference a mutable value that is mutated later,
even though the function isn't called during render).
Instead we use the same approach as validateNoSetStateInRender, explicitly
tracking references to function expressions that access refs, and checking if
those function expressions appear to be called. This can have false negatives,
as with the setState validation, but catches lots of obviously incorrect code
without false positives.
Fixes T178003134. Previously we did not check whether values reassigned during a
destructuring assignment were context variables. This would either miscompile,
or as of my fix earlier in #2579, would fail validation. Specifically, this
happened on AssignmentEpression with an object/array pattern lvalue, where the
pattern contained an identifier that is a context variable.
This is now fixed: we track whether the outermost assignment is a normal
assignment or destructuring, and force destructuring to a temporary whenever the
identifier is a context variable. We apply the same logic to variable
declarations that are destructuring to a context variable.
---
I recall adding the navigator override because some React library file had done
an unconditional access, but this doesn't seem to be the case anymore.
Regardless, newer versions of nodejs comes with a global `navigator` [see
thread](https://github.com/nodejs/node/issues/39540) that error on writes
Fixes the one case discovered in the previous PR; for AssignmentExpression we
correctly lowered the store instruction to a local/context, but then always used
a `LoadLocal` to read the result back.
The load instruction appears like it might be dangling - i think what was
happening is that DCE cleaned up the unused LoadLocal whereas it leaves the
LoadContext alone. But this works for now, we can always clean up the extra
instruction later since this case isn't too common.
Validates that all references to a variable (pre-SSA) are consistently "local"
references or "context" references. Ie, if a variable is declared as
DeclareContext, any accesses must be eg LoadContext or StoreContext, not
LoadLocal/StoreLocal. This will help with the issue from #2577 (assuming that we
know a variable _is_ a context variable) but also provides a more precise
bailout for an existing case with destructuring assignment to a context
variable.
Fixes T176436488. The logic for rewriting Destructure instructions was correct,
but the visitor implementation was accidentally dropping subsequent Destructure
instructions within a block after encountering one that needed a rewrite.
Switching to use the transform infra (added after this pass was written) fixes
it.
This is not that big a deal but a constant papercut, i often want to jump
directly to watch mode with a filter applied. I know @poteto likes to (or at
least used to) run watch with update enabled. Now instead of passing a mode, you
can pass `--watch`, `--filter`, and `--update` independently.
---
This change simply logs on every function we encounter with a `use no forget`
directive. A few nuances -- `compilationMode: "infer"` only compiles functions
we infer to be 'react functions'.
```js
// `add` would not be compiled, as it has no jsx, no hook calls,
// and is not named as a component or hook
function add(a, b) {
return a + b;
}
```
With this PR, we would report todos for functions that Forget wouldn't
ordinarily try to compile.
```js
// Todo: Skipped due to "use no forget" directive.
function add(a, b) {
"use no forget";
return a + b;
}
```
This seems fine to me as (1) it's a bit nonsensical to have a `use no forget`
direction on a non-react function, and (2) we're goalling on getting `use no
forget`s down to 0.
The goal of this PR is to move towards a uniform representation for all type
declarations, whether they are named type aliases, function declarations, or
inline annotations. We now assign every non-primitive type declaration (named or
anonymous) a unique DeclarationId. In the next PR, we'll also re-map inline
annotations back to this declaration id when encountering them.
This PR is extremely gross and my intent is to refactor a bunch of things in the
HIR to allow this to be less gross. Challenges:
* Babel name resolution requires using scopes but i really want to just work
with plain nodes, since NodePath and TypeScript do _not_ get along. So here, i
find all identifiers and store a mapping of identifier -> scope, so that i can
later look them up if necessary.
* HIR doesn't have a notion of a declaration id, and in general we don't want
to extend HIR. So i end up with a whole bunch of side table information and
indirection. For example, a function doesn't know it's own declaration id.
So we have to look it up. Function params don't track their Forest type, so
we have to look them up on the function declaration. Etc.
The current error message "This mutates a global or a variable after it
was passed to React" no longer makes sense since we now have more
specific error messages for different kinds of Effect.Mutate or
Effect.Stores. This replaces the fallthrough "Other" case with a
more generic message. It's not perfect, but it's a little more accurate
than what is currently emitted
The proper fix might be to treat functions as mutable objects and allow
the mutation, or special case `Function.displayName`. For now though
this PR just updates the message in the meantime so it's less
confusing.
We're doing some internal benchmarking using a lightweight bundler that @pieterv
wrote for experimentation purposes. It's designed to fully preserve Flow type
annotations so we can experiment with type-driven compilation and test out what
benefits we might get from "cross-module" compilation more easily (ie by just
bundling together a few modules so we can see them all as one).
However, the bundler renames local variables and imports, so that a reference to
`useMemo()` might end up as `React$useMemo()` or similar. This PR adds a flag to
tell the compiler that builtin hooks might be prefixed and resolve them
appropriately.
---
Currently, we error on non-hoisted identifiers in EnterSSA with a somewhat
cryptic message. This PR changes `BuildHIR` hoisting logic to find ALL hoistable
bindings, then error when we try to lower hoisting for unsupported declaration
types.
Two benefits to this refactoring:
- Dedups "unhandled identifier declaration" logic (previous to #2552 and this
PR, we did this check in three places).
- More explicit todo diagnostic messages when we cannot hoist a declaration
---
Three functional changes:
- Instead of visiting all identifier references, explicitly traverse only
function decls/exprs. This avoids bugs like accidentally hoisting inline
references
```js
// input
const x = identity(y);
const y = 2;
// lowered HIR before this PR (simplified)
[0] DeclareContext HoistedConst y$0
[1] LoadContext y$0
[2] StoreLocal Const x$5 = identity([1])
```
- Rely on `isReferencedIdentifier()` instead of manually checking member
properties / assignments, which is error prone
```js
// added fixture hoisting-repro-variable-used-in-assignment
const callbk = () => {
// before this PR, we skip hoisting x because it's part of a declaration
const copy = x;
return copy;
};
const x = 2;
return callbk();
```
- Visit lvalues after rvalues. This allows for recursive self-references (e.g.
factorial)
From the Babel side, this change relies heavily on babel's scope binding
resolution logic. My understanding is:
- Babel guarantees node objects are uniqued (`node1 === node2` <--> node1 and
node2 are the same node in the ast)
- Each binding has exactly one `bindingIdentifier` (`binding.identifier`,
`getBindingIdentifier`, etc) which is identifier node @ its declaration site
```js
// x is a binding identifier
const x = 2;
// foo is a binding identifier
function foo() {
}
// param is a binding identifier
(param) => {...}
// this bar is a binding identifier
let bar;
// but not this bar
bar = 2;
```
---
This is likely a rare edge case, but it does produce a parse error.
RenameVariables visits all identifier references to ensure we don't end up
producing conflicting variable declarations, using a stack of block scopes to
check "in scope variables".
This pass is currently built to be conservative -- we explicitly rename shadowed
variables, and visit all rvalue references. The issue is for this IR:
```
{
1. decl t0;
2. scope 0 {
3. reassign t0 = ...
4. read(t0)
5. }
6. let t0 = ...
7. read(t0);
}
```
We currently visit t0 only on line 4 and 7 (and never rename t0). Instead we
should visit lvalues (declaration sites) which occur earlier than rvalues
(visiting lines 1 and 6 will show conflicting declarations)
The compiler bails out of compiling code that contains suppressions of the
official React ESLint rules. However, some apps may use additional rules that
they want to trigger bailouts for, or use the official rules under a different
name (we do this at Meta). This PR adds a compiler flag to specify a custom set
of line rule names, suppression of which should trigger a bailout.
Fixtures from T173102122 and T173101739 demonstrating cases where
MergeConsecutiveBlocks can move code out of its correct block scope, changing
behavior or breaking the program, in cases where a control flow structure (such
as switch) only has one non-returning control flow path. In these cases, the
non-returning path gets merged with the fallthrough, effectively lifting that
code out of the control flow structure and moving it into the outer scope. This
can create dead code or just invalid code (with references to variables that are
not in scope).
Sprout fails on both of these fixtures:
<img width="812" alt="Screenshot 2024-01-23 at 11 25 36 AM"
src="https://github.com/facebook/react-forget/assets/6425824/d397ea22-3fa3-436e-b655-09a45781274b">
See the previous PR, interleaved mutation can cause values that were not
reactive to become reactive. I swear I had a case where this was observable, but
I came up with it before reordering the PRs in this stack. I think my repro
relied on an immutable reference to a mutable value, which is now handled in
InferReactivePlaces. So here i'm just adding fixtures, and allowing this case
since it's unobservable.
During PruneNonReactiveDependencies, we sometimes need to promote a value from
non-reactive to reactive if it ended up being grouped in the same reactive scope
as some other reactive value. This generally happens due to interleaving
mutations.
In this case all downstream usage of the promoted value need to also be
considered reactive. Fully propagating the reactivity requires re-running
InferReactivePlaces, to account for things like control reactivity. We can't yet
reuse that pass here though, because we haven't unified the pipeline on HIR yet.
For now, we propagate the reactivity through local variables and downstream
reactive scopes. See test fixtures for some examples that now correctly
propagate reactivity and some that need the full reactivity inference to run
correctly. The latter cases are handled in the next PR.
I found this by adding logic to reject inputs where reactivity gets newly
propagated in PruneNonReactiveDependencies. It's possible to create a readonly
alias to a mutable value such that we don't know the value is reactive yet when
the alias is created. Thus we need to do a fixpoint iteration even if there are
no loops in order to be able to revisit such aliases and reflow the reactivity
forward. Example:
```javascript
const x = [];
const y = x;
const z = [y]; // y isn't reactive yet when we first visit this, so z is
initially non-reactive
y.push(props.value); // then we realize y is reactive. we need a fixpoint to
propagate this back to z
const a = [z]; // need an indirection to get past the partial propagation in
PruneNonReactiveDependencies
let b = 0;
if (a[0][0]) {
b = 1;
}
return [b];
```
Existing fixtures don't change because the basic reactivity propagation in
PruneNonReactiveDependencies is enough to make common cases work. I confirmed
that the new fixture does not work on previous PR in the stack.
Fixes T175227223. When inferring reactivity, mutation of a value with a reactive
input marks the mutable value as reactive. However, we also need to account for
aliases:
```javascript
const x = [];
const y = x;
y.push(props.value);
```
Previously we would have only considered `y` reactive here, but `x` also becomes
reactive.
The implementation extracts out a helper from InferReactiveScopeVariables that
builds a `DisjointSet<Identifier>` of disjoint sets of mutably aliased values.
InferReactivePlaces then treats all instances of each mutable alias group as
equivalent for reactivity purposes.
In InferReactivePlaces, we already account for reactively controlled values:
where a value is never assigned a non-reactive value, but _which_ value is
assigned is based on a reactive condition (the test conditions of an if, switch,
loop, etc).
This PR extends that reactively-controlled inference to mutation that is
conditioned upon a reactive value. From the test case:
```javascript
let x = [];
if (props.cond) {
// This mutation has no reactive inputs.
// *But* the mutation conditionally occurs based on props.cond which is reactive
x.push(1);
}
let y = false;
if (x[0]) { // therefore the value observed here is reactive
y = true;
}
// so the value of y here is reactive via the reactive control dependency x[0]
return [y];
```
---
Previously, our logic was something like:
```js
fixed-point-loop {
foreach instruction {
mark referenced identifiers
// assume that usages are always visited before declarations
if (instruction is decl) {
prune(instruction);
}
}
foreach instruction {
if not referenced {
delete(instruction);
}
}
```
This contained a bug, as not all usages of a variable are guaranteed to be
visited before its declaration.
```js
// input
let x = 0;
while(x < 10) {
x += 2;
}
return x;
// hir
entry:
x$0 = 0
goto loop-test
loop-test:
x$1 = phi(x$0, x$2)
if ... goto loop-body else goto fallthrough
loop-body:
x$2 = x$1 ...
goto loop-test
fallthrough:
return x$1
```
In this example,`x$2` is defined by `loop-body` and used by `loop-test`.
Similarly, `x$1` is defined by `loop-test` and used by `loop-body`.
---
TODO: trying to come up with more test fixtures
Same babel identifier issue as #2510 but for HoistedConst
Not sure how we should best test this -- one possibility is using constant prop.
Currently, we have false positives for HoistedConst that prevent constant
propagation. I don't want to over-rotate on babel apis tests in our fixtures
(instead of semantically interesting ones)
```js
// input
function Component() {
{ x: 4 };
const x = 2;
return x;
}
// output
function Component() {
const $ = useMemoCache(1);
let x;
if ($[0] === Symbol.for("react.memo_cache_sentinel")) {
x = 2;
$[0] = x;
} else {
x = $[0];
}
return x;
}
```
identifiers
---
A few fixes for finding context identifiers:
Previously, we counted every babel identifier as a reference. This is
problematic because babel counts every string symbol as an identifier.
```js
print(x); // x is an identifier as expected
obj.x // x is.. also an identifier here
{x: 2} // x is also an identifier here
```
This PR adds a check for `isReferencedIdentifier`. Note that only non-lval
references pass this check
```js
print(x); // isReferencedIdentifier(x) -> true
obj.x // isReferencedIdentifier(x) -> false
{x: 2} // isReferencedIdentifier(x) -> false
x = 2 // isReferencedIdentifier(x) -> false
```
Which brings us to change #2.
Previously, we counted assignments as references due to the identifier visiting
+ checking logic. The logic was roughly the following (from #1691)
```js
contextVars = intersection(reassigned, referencedByInnerFn);
```
Now that assignments (lvals) and references (rvals) are tracked separately, the
equivalent logic is this. Note that assignment to a context variable does not
need to be modeled as a read (`console.log(x = 5)` always will evaluates and
prints 5, regardless of the previous value of x).
```
contextVars = union(reassignedByInnerFn, intersection(reassigned,
referencedByInnerFn))
```
---
Note that variables that are never read do not need to be modeled as context
variables, but this is unlikely to be a common pattern.
```js
function fn() {
let x = 2;
const inner = () => {
x = 3;
}
}
```
I sincerely appreciate the effort to get test262 up and running. This was my
idea, it seemed like a really good way to test our correctness on edge cases of
JS. Unfortunately test262 relies heavily on a few specific features that we
don't support, like classes and `var`, which has meant that we never actually
use this as a test suite.
In the meantime we've created a pretty extensive test suite and have tools like
Sprout to test actual memoization behavior at runtime, which is the right place
to invest our energy. Let's remove?
Do we still use these? I'm happy to close this PR if we still want this but it
feels like these may have served their purpose and no longer be necessary.
Fixes the false positive in the previous PR. When we prune a scope because it's
values are non-escaping, we now also remove any `Memoize` instructions for that
scope. The intuition being that we're actively removing unnecessary memoization,
so we don't need to check that the memoization occurred anymore.
This demonstrates a false positive in validatePreserveExistingManualMemoization.
We prune memoization of non-escaping values, but the validation pass just sees
that the value "should" have a scope and that scope doesn't exist, and thinks we
failed to preserve memoization.
Interpolating values into the `reason` field of an error breaks our error
aggregation. This PR moves the offending function name into the `description`
field which isn't used for aggregation.
I had trouble checking out the repo using Sapling because the submodule couldn't
clone properly. I got
> Error: Permission denied (publickey)
In my branch I tested that the https URL format seems to work okay with Sapling.
Add frozen reason for props and hook arguments
Improves the error message when mutating props or hook arguments.
Previously, this would print a generic error about mutating global variables.
This was an oversight in the original definition of useContext (oops my bad).
Context values are owned by React and should not be modified. I found this
because some cases of existing useMemo were not preserved (tested via the
validatePreserveExistingManualMemo flag) due to function calls referencing
context being assumed to mutate.
This change will allow more memoization, it's also just more correct for the
rules of React. Note the new ValueReason variant so that we can provide a
precise error message about mutating context values.
It's starting to get complex just with a couple of extra
passes — we either need to substantially extend the HIR or (as i've done so far)
pass information from early passes to later ones. This PR changes things so that
very early in the babel plugin we fork into a separate mode. Forest has
its own `compileProgram()` equivalent, its own pipeline, its own codegen, etc.
> Update: this is now passing all tests. The approach is likely wrong, and even
if it's fine it needs some cleanup. Putting up for review as folks (esp
@gsathya) have time.
## Background
InferTypes was intended to infer types for phi identifiers, but by accident we
ended up storing the inferred type on `phi.type` instead of `phi.id.type`, which
is the type that usages of the phi will reference. Because of this, we weren't
actually inferring types for several cases, for example if both if/else branches
assign `x` to an array literal, we'd ideally like the corresponding phi id to be
typed as a BuiltInArray:
```javascript
let x;
let y = { ... };
if (cond) {
x = [];
} else {
x = [];
}
// x should be BuiltnArray here. We inferred that on Phi.type but the x here
wouldn't get that type previously
x.push(y);
```
## Circular Types
I started by removing the `Phi.type` property and updating inference to store
the result of phi unification on `phi.id.type` — but this revealed other issues.
First was this can create circular types when there are loops. The solution is
to basically allow circular types _for phis only_, and when we detect them we
remove the cycle. Basically whenever we have a situation where we have some type
variable X, and a type Y that is a (nested) phi type one of whose transitive
operands contains X, we remove X from the transitive type and attempt to
collapse the phi type upwards if all of its remaining operands are the same:
```
X=Type(1)
Y=Phi [
Type(2),
Type(3) = Phi [
Type(1), // <-- cycle but we can prune this
Type(2),
Type(2),
]
]
=>
X=Type(1)
Y=Phi [
Type(2),
Type(3) = Phi [ // all remaining operands are the same, we can prune this
Type(2),
Type(2),
]
]
=>
X=Type(1)
Y=Phi [ // all remaining operands are the same, we can prune this
Type(2),
Type(2),
]
=>
X=Type(1)
Y=Type(2)
```
We have to do this not just doing unify(), but also in `get()` since there are
cases where we don't know yet which type variables we can remove from a phi.
Without also doing the pruning in get, we get an infinite loop.
## Reactive Scope Alignment
The above fixed the circular types, but exposed some new cases that can occur in
terms of mutable ranges and ast structures: it wasn't possible before to have a
Store on a phi node in practice, since that relied on type information which we
didn't have for phis.
The new validation that all instructions for a scope are part of that scope
caught a couple issues, which were basically like this:
```
[1] Sequence
...
[9] StoreLocal x@0[9:28]
[10] ...
```
Note that scope 0 starts at instruction 9, but that instruction is not at the
block scope level. The first instruction at the block scope level that is within
the range of scope 0 is instruction 10, which is after the scope should have
started! So I also had to update AlignScopesToBlockScopes to handle the case of
logical, conditional, and sequence expressions: we sometime need to adjust a
scope start earlier in case they contain instructions that should start a scope.
Fixes the case from the previous PR by using a different sentinel for
uninitialized cache values and early returns. I confirmed with console.log that
the reactive scope for `x` only evaluates on the first execution, after which we
figure out that we don't need to execute it again.
RFC. This is a quick sketch of adding support to Sprout to render the same
component instance multiple times with different props. This doesn't test
memoization (though it forms a basis for testing it, more below), but does allow
us to test that the code properly reacts to inputs and doesn't get "stuck"
always returning the same output even when inputs change.
Possible extensions:
- Support calling non-component functions multiple times
- Test memoization by having the `toJSON()` helper track objects it has
encountered before, assign each object a unique id, and then emit subsequent
references to the same value as the id instead of the printed form of the
object.
For example if we call a memoized function with the same input twice in a row,
today we might get output like:
```
[{a: 1}],
[{a: 1}],
```
Which doesn't tell us if the object is equal. Instead we could emit output like:
```
[{a: 1}] #0,
#0,
```
Which allows verifying that memoization actually happened. Or we could automate
this and just assert that anything structurally equal has to be referentially
equal — though there are cases with conditionals that break this.
Adds support for early returns within reactive scopes, behind a new feature
flag. The flag is off by default, where this case continues to throw a Todo
bailout.
Since implementing a sketch of the codegen in the previous PR I realized that
it's easy enough to implement the more optimal output, so i've updated that
here. Rather than both the if and else branch of the reactive scope having an
"if the return value was not a sentinel return it" check, we instead make the
return temporary a proper declaration of the reactive scope. Then, since it's
actually an output it's available in the outer block scope, and we could do a
single if-return after the reactive scope.
Edit: see comment below for thoughts on test cases.
Implements codegen for reactive scopes with early returns, though we don't ever
construct such a case yet. See comments in the code. There is a slightly more
optimal output that would require a larger refactor (also described in code
comments), for now i'm starting with the simpler approach since this is
relatively rare so we don't need to optimize code size / runtime as much.
Adds a new `earlyReturnValue` property on ReactiveScope which will be set if the
scope had one or more early returns, with information about the temporary
identifier that the early return value will be assigned to, as well as the label
to be used for breaking (to simulate the early-return). The next PR shows the
intended codegen.
Adds a new compiler pass that will eventually actually handle early returns
within reactive scopes. For now it just detects them and throws a Todo error.
Adds back a mode to transitively freeze function expressions, independently from
the mode to preserve existing manual memoization. This lets us experiment with a
few variants:
* Preserve existing memoization
* Validate existing memoization with:
* `enableAssumeHooksFollowRulesOfReact` &&
`enableTransitivelyFreezeFunctionExpressions`
* `enableAssumeHooksFollowRulesOfReact` only
* neither of those flags
Note that `enableTransitivelyFreezeFunctionExpressions` alone probably doesn't
make sense, it's more aggressive than
`enableAssumeHooksFollowRulesOfReact` so we might as well try them together.
Adds a new mode which validates that existing manual memoization is preserved
_without_ using information from the manual memoization to affect compilation.
This gives us a way to try out the more aggressive version of Forget — ignoring
manual memoization — first and see how much code bails out and what patterns
cause this.
We can then proceed to enable the mode to actually _preserve_ existing memo
guarantees only where necessary.
Extends `@enablePreserveExistingMemoization` to validate that all of the
original values were actually memoized. This works nearly identically to how we
validate effect deps are memoized. We look for Memoize instructions whose values
need memoization but whose range extends past the memoize instruction, or where
the value isn't memoized at all.
Merges `@enableTransitivelyFreezeFunctionExpressions` into the new
`@enablePreserveExistingMemoizationGuarantees` mode, since they are both
motivated by the same use case of preserving effect behavior by preserving
existing memoization behavior.
The idea is that `useCallback` has an implicit assumption: that the variables
captured by the callback aren't subsequently modified. Previous PRs treated the
values directly captured by the callback as frozen. But if those variables were
themselves another function expression, and that expression captured a mutable
value, then we wouldn't consider the freeze to be transitive:
```javascript
const object = makeObject();
useHook(); // oops, hook call inside `object`'s mutable range, can't memoize
object, log, or onClick!
const log = () => { console.log(object) };
const onClick = useCallback(() => { log() });
maybeMutate(object);
```
However, the assumption of such code is that it _doesn't_ modify such
transitively captured values. So here we merge
`@enableTransitivelyFreezeFunctionExpressions` mode into the
memoization-preserving mode. Now, the memoize instructions emitted for
useCallback (and useMemo) will transitively freeze captured function
expressions, allowing us to memoize.
The flip side of this is that some code may be violating these rules. We'll rely
on runtime validation to detect such cases.
Adds test cases per the previous PR for useCallback:
* callback that references another callback, which in turn references a
possibly-mutated value
* callback that references a ref
Improves `@enablePreserveExistingMemoizationGuarantees` for the useCallback
case. Similar to useMemo, we add an explicit `Memoize` instruction for the
callback function itself _and_ for its dependencies. This means we'll assume the
callback doesn't mutate any captured variables.
TODO: check this with cases involving refs (should be allowed, but also not
accidentally freeze the ref) and reassignment of locals (should be disallowed,
though that might just be a validation we're missing today)
The previous PR introduced `memoize` instructions whose lvalues aren't used, but
which can't be pruned by DCE due to pipeline ordering. Here we change to make
memoize an instruction intended for its side effects only, and prune during
codegen.
See discussion on #2448 for full context. In the new
`@enablePreserveExistingMemoizationGuarantees` mode, the goal is to preserve the
existing referential equality guarantees from the original code. #2448 lays the
groundwork by explicitly marking the _output_ of each useMemo block as memoized,
hinting to the compiler that the value cannot subsequently change. This ensures
the mutable range doesn't extend _later_, possibly overlapping a hook call and
causing memoization to gett pruned.
This PR fixes the other direction. There are cases where free variables
referenced in the useMemo block could have been inferred as mutated, which could
then extend the _start_ of the range earlier past a hook:
```javascript
const foo = createObject();
useBar();
const baz = useMemo(() => {
const baz = createObject();
maybeMutate(foo, baz);
return baz;
}, [foo]);
```
Here the compiler would infer that both `baz` and `foo` are mutable at the
`maybeMutate()` call, grouping them in the same scope. But that scope would span
the `useBar()` call, and be pruned, meaning that `baz` went unmemoized.
However, useMemo blocks shouldn't be mutating free variables. Only variables
newly created within the useMemo block should be mutable. So this PR extends the
feature to treat all free variables referenced in a useMemo block as frozen as
of the block itself.
Adds an option to preserve existing memoization guarantees for values produced
with useMemo and useCallback. We still discard the calls to these hooks, but we
preserve the information that the value is frozen at that point in the program.
Because these values are produced solely within the useMemo/useCallback
callback, their mutation cannot have any interspersed hook calls. This means
that the values mutable range will never span a hook and end at the point of the
useMemo, ensuring that they are memoized at the same point.
The main things that can change (relative to the orignal code) are:
* Forget will infer a precise set of dependencies, ignoring the user-provided
values. In practice this should only occur if the original code had a lint
violation, which Forget would bail out on. So in practice this shouldn't happen
unless the code doesn't use the React linter.
* Forget may start the memoization block earlier than the developer did if other
values are mutated along with the value being produced. This can cause
memoization to fail, but only in situations where it would have failed
previously:
```javascript
const a = [];
useFoo();
const b = useMemo(() => {
const c = a;
c.push(1);
return c;
}, [a]);
```
In this example (sans Forget) the useMemo will invalidate on every render
because `a` will always be a new array and its listed as a dependency of the
useMemo. Forget would correctly determine that the memoization would have to
work as follows:
```javascript
let c;
if (...) {
const a = []
useFoo(); // OOPS we made a hook call conditional
const t0 = a;
t0.push(1);
c = t0;
...
} else {
c = $[...]
}
```
Because this is invalid, Forget would (later in the pipeline) strip out this
memoization block and (as with the original) leave `c` un-memoized.
In this same example, removing the hook would cause Forget to be able to memoize
a value that wasn't memoized before:
```javascript
const a = [];
const b = useMemo(() => {
const c = a;
c.push(1);
return c;
}, [a]);
```
This invalidates every render without Forget, but would memoize correctly with
Forget (it would expand the memoization block to include the declaration of
`a`).
Adds a fixture for our existing behavior that reactive scope dependencies
exclude values which are non-reactive. The idea is that regardless of whether
the value may actually get recreated over time or not, a "nonreactive" value
cannot semantically change and therefore we can ignore changes in its pointer
address.
After running the latest hook validation internally, I found some cases where
there was a violation but the error message was not ideal. For example on this
code:
```javascript
usePossiblyNullHook?.();
```
We reported a "hooks can't be used as normal values" violation, when we'd
ideally report a "hooks can't be called conditionally" violation. The solution
in this PR is to track errors by source location, and upgrade the former
violation to the latter, more serious violation. See fixtures for examples.
ObjectExpressions
---
Currently, we're removing all reactive scopes containing object methods. This
could produce incorrect output as object method instructions may still be
included in other reactive scopes (and will lose their dependencies).
Builds on the utilities added previously to infer types from type annotations on
variable declarations. This is a limited form, where currently we only infer for
local identifiers (not function parameters) and only infer a type for the
variable initializer and not subsequent reassignments.
This PR uses the information from type cast expressions (`as` or `(variable:
type)`) to inform type inference. BuildHIR converts the type annotation to our
internal type format where possible, falling back to the generic `makeType()`.
This is then used in InferTypes to help set the value's type.
Extends the previous analysis to work for PropertyLoad and ComputedLoad, so that
if the object is a prop we track the resulting value as a signal. We also
disallow PropertyLoad/ComputedLoad outside of a reactive scope where the object
is a signal (since that would drop reactivity).
Previously the only way to replace a value was to override transformInstruction
and transformTerminal, and to be careful to find nested values. This PR adds
`ReactiveFunctionTransform#transformValue()` which allows returning an optional
new value, which if present will replace the value in whatever context it
appeared. ReactiveFunctionTransform now reimplements all the methods necessary
to replace any value anywhere in the AST. See the next PR for an example usage.
I realized this while working on Forest. When computing the dependencies of a
reactive scope we can omit setState functions in the general case (exception
described below). Currently that's implemented in PruneNonReactiveDependencies.
However, this causes us to miss some optimizations — a value isn't reactive if
its only dependency is a setState, and that may allow further downstreams values
to become non-reactive. We lose out on that by only filtering out setStates in
PruneNonReactiveDependencies — this logic really belongs in InferReactivePlaces.
So this PR moves the check for setState types to that pass. The updated fixtures
show that this already uncovers some wins. The _new_ fixtures covers the
exception. It's possible for a value to be typed as being a setState function,
but to still be reactive: if its a local that is conditionally assigned
different setState function values. Currently this test happens to work because
our phi type inference is incomplete (see #2296). I'm adding the test now though
to prevent regressions when we fix phi type inference.
In normal React certain operations don't allocate new objects (property loads,
binary expressions, etc) and therefore don't need a reactive scope in Forget.
For example, property loads only extract part of an existing value and don't
allocate something new, while binary expressions are known to produce primitive
values that don't allocate. We rely on the fact that whenever their inputs
change we will re-run the component/hook and propagate the result forward.
For Forest, the only way to propagate data is via reactive scopes: the component
code is equivalent to a "setup" function. This PR updates some of our passes to
ensure that we create (and don't prune) scopes for these types of operations. I
started with a conservative set for now.
The previous PR converts reactive scopes to normal instructions, so that Forest
mode won't have any scopes left by the time we reach codegen. This PR removes
the now-unused codegen logic for forest.
For Forest, we previously converted reactive scopes into derived signals during
Codegen. I'm moving this to a separate pass primarily to keep codegen simple
since there's enough complexity just dealing with core JS semantics. Ideally
we'd do a similar setup even for regular Forget, ie lower reactive scopes just
prior to codegen.
At the same time i also reordered the forget passes to be just before codegen,
and cleaned things up a bit. For state lowering, we now just rewrite `useState`
-> `createState`, because we actually need to keep around the setter function to
trigger scheduling updates in addition to writing the signal value.
Found from eslint validator on www after doing a local sync + RunForget test of
#2432
P898168203
> New Errors:
> no-undef:'VARIABLE_NAME' is not defined.
---
I modeled guards as try-finally blocks to be extremely explicit. An alternative
implementation could flatten all nested hooks and only set / restore hook guards
when entering / exiting a React function (i.e. hook or component) -- this
alternative approach would be the easiest to represent as a separate pass
```js
// source
function Foo() {
const result = useHook(useContext(Context));
...
}
// current output
function Foo() {
try {
pushHookGuard();
const result = (() => {
try {
pushEnableHook();
return useHook((() => {
try {
pushEnableHook();
return useContext(Context);
} finally {
popEnableHook();
}
})());
} finally {
popEnableHook();
};
})();
// ...
} finally {
popHookGuard();
}
}
// alternative output
function Foo() {
try {
// check current is not lazyDispatcher;
// save originalDispatcher, set lazyDispatcher
pushHookGuard();
allowHook(); // always set originalDispatcher
const t0 = useContext(Context);
disallowHook(); // always set LazyDispatcher
allowHook(); // always set originalDispatcher
const result = useHook(t0);
disallowHook(); // always set LazyDispatcher
// ...
} finally {
popHookGuard(); // restore originalDispatcher
}
}
```
Checked that IG Web works as expected
Unless I add a sneaky useState:
<img width="705" alt="Screenshot 2023-12-05 at 6 44 59 PM"
src="https://github.com/facebook/react-forget/assets/34200447/3790bd76-7d71-44b5-a62e-f53256fb5736">
---
Prior to this PR, we were mutating functions after CodegenReactiveFunction
completes (in `Entrypoint/Program.ts`).
The reasoning for this separation was that we wanted to keep non-compiler logic
out of the core Pipeline. However, it made our code difficult to read and reason
about.
Open to other alternatives, like adding a pass after Codegen.
---
Currently on main, rollup does not inline source files
```js
// in packages/babel-plugin-react-forget
// $yarn build
// output
var CompilerError_1 = require("./CompilerError");
Object.defineProperty(exports, "CompilerError", { enumerable: true, get:
function () { return CompilerError_1.CompilerError; } });
// ...
```
I debugged a bit but not familiar with node or rollup.
- It seems that rollup fails to recognize source file imports with this setting,
as resolveId no longer gets called
- current `module` option defaults to `ESNext`, which works for some reason.
Let's revert for now to unblock syncs.
Sanity checked my repro by reinstalling node-modules and cleaning rollup cache.
New approach to hooks validation per recent discussion. The idea is to avoid
false positives while still preventing serious violations. See the comments in
the file for more details about the approach. It uses a somewhat similar idea to
InferReferenceEffects in that we track a "Kind" for each IdentifierId, and
various instructions propagate or derive a result Kind from the operands. Kinds
form a lattice and can be joined, allowing us to be more precise about known vs
potential hooks, and known vs potential _sources_ of hooks.
The previous PR helped me realize we weren't handling Array#at correctly. If the
receiver is a mutable value its effect should be Capture and the lvalue effect
needs to be Store. This PR updates the definition for Array#at to make the
receiver Capture, and then updates inference to automatically set the lvalue
effect to Store if _any_ argument (or the receiver) was Capture.
There was one missing piece to the optimization from the previous PR: Array#map
can return an alias to the receiver in its output, which means that mutations of
the result have to be treated as mutations of the receiver. This means we need
to use a Capture effect on the receiver. If that doesn't get downgraded to a
Read bc the value was immutable, we then also need to make the lvalue effect a
Store (so that InferMutableRanges actually looks at it for aliasing).
Improves memoization for cases such as #2409:
```javascript
const x = [];
useEffect(...);
return <div>{x.map(item => <span>{item}</span>)}</div>;
```
We previously thought that the `x.map(...)` call mutated `x` since its kind was
Mutable. However, in this case we can determine that the map call cannot mutate
`x` (or anything else): the lambda does not mutate any free variables and does
not mutate its arguments.
This PR adds a new flag to function signatures, used for method calls only, that
checks for such cases. The idea is that if the receiver is the only thing that
is mutable — including that there are no args which are function expressions
which mutate their parameters — then we can infer the effect as a read. See
tests which confirm that function expressions which capture or mutate their
params bypass the optimization.
Distilled repro of an internal example we found. Forget determines a mutable
range for the array, but that mutable range spans a hook call, so the reactive
scope gets pruned. That's all working as expected.
What isn't ideal though is that if we know `x` is an array and `f` can't mutate
its arguments, then `x.map(f)` shouldn't count as a mutation of `x`, since
Array.prototype.map can only mutate the receiver via the callback (if the
callback mutates its args).
Improving on this example requires a) we have to know it's an Array, via type
information or bc we saw an array literal and b) being precise about which
functions could possibly mutate their parameters, which is tricky because of
indirect mutations via stores, etc.
We were using `returnValueKind` from function signatures for CallExpression but
not MethodCall; this PR changes to use this signature information for both
instruction kinds.
---
Going to hold off on landing until after codefreeze, it's not urgent as we
already fixed playground in #2404. All other internal pipelines do error
handling through Entrypoint, which catches and creates UnexpectedErrors as
needed.
Instead of using the source location to check for hoisting, just stop checking
for a given component after we reach it's declaration.
By definition all references to it before are (potential) hoisting errors.
Note that there could be false positives but that's ok.
Extends the validation that effect deps are memoized to handle an additional
case that @gsathya pointed out: when a dependency has a reactive scope but that
scope ends up being pruned. We track reactive scopes which actually exist in the
ReactiveFunction, and reject useEffect deps that have an associated reactive
scope but where that scope does not exist (bc it got pruned).
This is one approach to testing whether useEffect dependencies are memoized. The
idea is based off the observation that the only reason dependencies wouldn't be
memoized (other than compiler bugs) is that they are mutated later. If they're
mutated later, then the dep array will have a mutable range which encompasses
the InstructionId of the useEffect call. So we look for that pattern and throw a
validation error.
The downside of this approach is that we might reject code that happens to be
valid: specifically, that the lack of memoization isn't a problem in practice
because the effect won't trigger a loop. But (per test plan) this doesn't seem
to introduce that many new bailouts on www. Rather than implement a complex
validation that checks whether we un-memoized something that was memoized in the
input, it seems more practical to:
1. Enable this more comprehensive validation against any form of un-memo'd
effect dependency
2. Flip the default for hooks (to assume they follow the rules), which will fix
the primary cause of Forget pessimistically not memoizing dependencies.
## Test Plan
Synced to www and checked output via the upgrade script: a few components stop
getting memoized bc they have un-memoized effect dependencies. Let's chat!
We were modifying the Babel AST as a shortcut to lowering function declarations,
instead we can explicitly lower them equivalently to a `let <id> =
<function-expression>`.
When you have your panic threshold set to "NONE" as we recommend, it's easy to
miss that your config is wrong (which makes everything not compile) because
those errors were being silenced. This made debugging FluentUI and the
forget-feedback testapp pretty difficult to figure out at first, and defeats the
purpose of having config validation in the first place.
This pr makes it so InvalidConfig errors always throw, regardless of the panic
threshold set. In general our plugin should never throw at build time due to
component bailouts, but because an InvalidConfig would bailout everything from
being compiled at all, it seems reasonable to throw here
Babel doesn't attach Comment nodes to anything, so they dangle off of
the Program node while only specifying a range. This meant that
previously we first had to traverse all of the Program's comments to
find an eslint suppression of the rules of React, then during traversal
of the individual functions, we would check if there were any *global*
eslint suppressions, then bailout all components.
This PR updates our logic to determine if individual functions are
affected by an eslint suppression range:
- If an eslint suppression range falls within its body; or
- If an eslint suppression wraps the function
---
16 out of ~150 recently added sprout fixtures have exceptions that reflect
easy-to-miss mistakes in the fixture input, like forgetting to import
`useState`.
This PR adds snapshot files for sprout to prevent these mistakes (or catch them
at diff review time). This describes what it implements, but happy to take other
suggestions as well!
1. One sprout snapshot file for each input.
This significantly increases the number of files we have, but makes it clear
what the fixture is testing. I was hoping to expand on these snapshots to record
the result of multiple re-renders, or mounts (with different parameters), but
the tradeoff is that adding / changing fixtures will now require running `yarn
sprout --mode update`.
Some alternatives I've considered:
- Only record snapshots for fixtures that error (`result.kind == "exception"`).
This change would be pretty easy. This doesn't help sanity check sprout results
for general mistakes though ("am I testing what I want to test?)
- Warn loudly for fixtures that error -- seems like these are easy to ignore,
but.. worth it for the convenience?
- Some github action that runs and comments on your PR with sprout fixture
changes; i.e. never manually update sprout files?
2. Sprout and snap share a common snapshot file, with some parsing hackery.
Previously, the implementation added new files to a separate sprout snapshot
directory, assuming that devs don't need to read these often.
After feedback from @josephsavona, I updated to have snap and sprout write to
the same file for ease of reviewing / debugging (to be able to see the input /
output side by side).
- `snap --mode update` will update snap's part of the file.
- `sprout --mode update` will update sprout's part
- `snap` and `sprout` will both compare oldSnapshot (read from the file) with
`newSnapshot = merge(oldSnapshot, newData)` to determine if a test pass.
3. Some hacks 😅 Absolute paths to project directory (e.g. stack traces) are
replaced with `<project_root>` in a mocked `console.log`
---
jsdom and other libraries seem to cause jest workers to exit with
`forceExit:true`. Not sure what option I set (or global I've overwritten) but
console logs aren't flushed as a result, making debugging a bit confusing.
This PR:
- Moves some code from `eval(...)` to a typechecked real js function. I always
had trouble debugging the `eval`ed code, so smaller code snippet is better here.
- waits for jest workers to end before exiting
I ran the plugin with the extended version of ValidateNoSetStateInRender enabled
(incl. function expressions) and there are no false positives. Let's remove the
flag for the function expression case since the whole rule is working
accurately.
Repro from T169063835. This works, but since it came up it seems good to add a
test case for it just in case there's something funny here that we could regress
on w/o realizing it.
Fixes one category of bugs with const hoisting. The algorithm finds all consts
that need to be hoisted, then looks through the statements of a block to find
the first statement which references that const, delaying the emission of the
HoistedConst instruction until its actually used. To determine if a statement
references a const we find every identifier in the statement and check if its
binding is one of the hoisted bindings.
There's a very small bug here: when we resolve the binding of each identifier,
we need to resolve it in its own scope. We're currently resolving these
identifiers agains the outer block statement's scope, which can cause us to
misattribute identifiers when there is shadowing:
```
const items = props.items.map(x => x); // we scan this statement, resolve 'x' in
the block statement scope, and mis-attribute it to the outer x.
const x = 42; // (1) this x is a candidate for hoisting, so the binding is in
the set of hoisted consts
```
I had added a repro for this earlier but hadn't realized it was due to const
hoisting. Renaming this test to clarify what's causing the problem and to make
it easier to find.
This is non ideal but at least it's a step in the right direction.
Getting the correct error requires us to track every identifier and global,
which seems a bit excessive for now.
We can revisit and improve this error if this is starting to confuse folks.
---
We were throwing `InvalidReact` errors on valid inputs.
```js
// Input
function Foo() {
log("block0");
for (const _ of foo) {
log("loop");
}
useBar();
}
// IR
bb0:
// log("block0");
ForOf init=bb2 loop=bb3 fallthrough=bb1
bb2:
// init
Branch: then:bb3 else:bb1
bb3:
// log("loop")
Goto(Continue) bb2
bb1:
// useBar();
Return
```
We correctly compute post dominators here.
```js
// In validateUnconditionalHooks
console.log(dominators.debug());
/* Output:
(read x => y as x is the post-dominator for y (all paths from x to the exit must
go through y))
"bb4" => "bb4",
"bb1" => "bb4",
"bb2" => "bb1",
"bb3" => "bb2",
"bb0" => "bb2",
*/
```
However, `findBlocksWithBackEdges` prevented us from adding `bb1` to the
`unconditionalBlocks` set as `bb2` (its post dominator) has a back edge. I'm not
sure what the `findBlocksWithBackEdges` was doing previously, so I replaced it
with an invariant asserting that the loop terminates.
---
Snap should compile all fixtures to record changes in results, even `todo`
prefixed ones. Previously, they were skipped as we noted the correlation of `//
@skip` pragmas and file naming.
Now, no fixture should be skipped as our compiler pipeline should be able to
handle every kind of error.
Noticed from our paste that we weren't correctly rolling up hoisting related
errors due to specific information being in the error title, so this PR moves
them into description instead.
Updates the approach used in ValidateNoSetStateInRender to detect function
expressions called during render. We now do the following:
* Track function expression which are known to unconditionally call setState
themselves- if these functions get called, that’s equivalent to calling
setState. We call the validation recursively to compute this.
* Track LoadLocal/StoreLocal indirections for such function expressions.
* Check CallExpressions where the callee is either a known SetState (via type
info) _or_ (new) where the callee is in the set of known-to-setState function
expressions.
The Set is shared throughout the analysis, so we can even find multiple levels
of indirection (see new test case).
I found an interesting edge case in the previous diff with mutation of a value
that appears in the expression of an object key:
```javascript
const key = {}
const object = {
[mutateAndReturnOtherValue(key)]: 42,
};
mutate(key);
```
We analyze and represent this correctly all the way through to codegen, but then
we hit the bug that @mofeiZ has noticed before: the temporary for `t =
mutateAndReturnOtherValue(key)` isn't emitted immediately (bc its a temporary).
It gets emitted inside the memo block for `object`, which is incorrect.
I tried to reproduce that here with JSX and it works as expected. It's an
interesting case though so let's land this to ensure we don't regress.
Adds a separate compiler flag for enabling the incomplete validation of ref
access within function expressions. Unlike the previous PR for
set-state-in-render validation, ref access in render can be okay in some
circumstances so i'm leaving this off by default. The point of splitting this up
is that our linting will be able to enable the rule without risk of false
positives.
The approach i initially took to validating function expressions was to try to
extend the mutable range if they are called during render, and then use the
mutable range of a function to determine if it's called during render later.
However there are cases where the range can be extended for other reasons, as
@poteto discovered, so we can't rely on the range extension. We've had several
of our validations completely off as a result of this.
In this PR i'm re-enabling @poteto's ValidateNoSetStateInRender pass by default,
but making the function expression checking use a separate compiler flag. This
means we'll have some false negatives, but should guarantee that we avoid false
positives. This means we can definitely catch things like:
```
const [state, setState] = useState(false);
setState(true);
```
Which we would have allowed by default before.
This reverts commit 10d129a8406e9d226abdb6943bf8512e34ce91db
---
Reverts #2311 due to undocumented assumptions being broken. I also added some
comments to `LoggerEvents` to explain each event type.
In `Program.ts`, we have something like the following code. `compile` could
produce any number of errors (not just expected errors / instances of
`CompilerError`). As an example, we sometimes error in `Codegen` due to babel
version incompatibilities (`Error: ObjectMethod: Too many arguments passed.
Received 7 but can receive no more than 5`).
```js
try {
// any error could be thrown here
compile(input);
} catch (e) {
// unknown type for e
handleError(e, ...);
}
```
I experimented with more variations but prettier collapses most of them: any run
of consecutive whitespace within a jsxtext node will get collapsed into a single
space, for example. So the main difference in practice is whether a jsxtext node
(preceding a value) has a trailing space/newline or not:
```
Text <fbt:param /> Text
// vs
Text<fbt:param />Text
```
which we now have tests for.
Turns out this will cause an OOM in node.js when running eslint as part of the
IDE
``` Before: 16M packages/eslint-plugin-react-forget/dist/index.js After: 2.2M
packages/eslint-plugin-react-forget/dist/index.js ```
Anytime we have a nested `.scope` in code my brain hurts. For example
`scope.scope.dependencies`. This PR updates the scope merging pass to use the
name `scopeBlock` for a ReactiveScopeBlock and `scope` only for ReactiveScope
values, to make things a bit more clear.
Addresses T168684688 (#2242). MergeReactiveScopesThatInvalidateTogether does not
merge scopes if their output is not guaranteed to change when their inputs do.
So for example a case such as `{session_id: bar(props.bar)}` will not merge the
scopes for `t0 = bar(props.bar)` and `t1 = {session_id: t0}`, because t0 isn't
guaranteed to change when `props.bar` does, and we want to avoid recreating the
t1 object unless it semantically changes.
But there's a special case: if a scope has no dependencies, then we'll never
execute it again anyway. So it doesn't matter what kind of value it produces and
it's safe to merge with subsequent scopes:
```javascript
return {session_id: bar()}
```
Without the reactive input, `bar()` will always return the same value since
we'll only ever call it once anyway. So it's safe to then merge with the scope
for the outer object literal.
Current sprout output (if you remove the line in `SproutTodoFilter`):
```
Failures:
FAIL: bug-fbt-preserve-whitespace
Difference in forget and non-forget results.
Expected result: {
"kind": "ok",
"value": "Before text hello world",
"logs": []
}
Found: {
"kind": "ok",
"value": "Before texthello world",
"logs": []
}
```
`fbt` transforms run before jsx ones, so our lowering should also account for
[fbt's whitespace
rules](https://github.com/facebook/fbt/blob/main/packages/babel-plugin-fbt/src/fbt-nodes/FbtImplicitParamNode.js#L230-L233)
in `BuildHIR:trimJsxText`.
Fbt + typescript [seems](https://github.com/facebook/fbt/issues/49) [to
be](https://github.com/facebook/sfbt/issues/72) a non-blessed workflow. We do
want to allow for both flow and typescript tests, so I followed some
instructions [from a
guide](https://dev.to/retyui/how-to-add-support-typescript-for-fbt-an-internationalization-framework-3lo0)
to add support.
- fbt tags desugar to.. "fbt" strings. By default, typescript removes unused
imports at parse step (and on autoformat steps). We pass special
babel-typescript configs and change vscode settings to mitigate this.
- I tried adding `fbt` to the global scope, but the fbt transform asserts that
`fbt` is actually imported in the source program.
- Other hacks are available, like saying we'll only allow for fbt in flow files,
or always patching the source code to have an "fbt" import. This seemed the most
reasonable and easiest to debug / follow when writing tests
---
Refactor selection logic to be easier to read; add support for .jsx test files
(feels a bit weird adding a `.jsx` fixture and not seeing it get run)
In order to make changes to the testapp's dependencies, we need to update the
lockfile which means modules need to actually exist. This adds a new script to
just run a build of the packages we sync so that module resolution in the
testapp will work
Reusing DEFAULT_HOOKS instance is a bit scary in case we mutate it by mistake
and this gets reflected across all compiles.
This isn't a concern now as we can't change the config during compilation. But
this PR keeps us safe in case we change this behavior in the future.
The tradeoff is a bit of perf which I think is the right tradeoff here.
I did a double take when I thought we didn't handle returning the
error when reading the code and when I edited the code, typescript told
me that there's no need to return as creating the error will throw.
This PR makes it clear from the name of the function that we will throw.
We should only add imports if we actually compiled anything, this is what caused
the internal issue despite the file in question not having any functions
opted-in to compilation.
When an `environment` isn't explicitly provided by the user's config, we used to
default this to `null` in `parsePluginOptions` which is called right at the
start of the Babel plugin. These parsed options were then being passed to zod,
which was expecting an object type, not null.
Zod can reify a default config based on the schema, if an empty object is passed
in. So by changing our default value to `{}` this should fix the compiler
bailing out incorrectly on valid compiler options
Test plan: tested this manually on the external repo by modifying node_modules
Use zod to do runtime validation and throw if incorrect.
This PR only adds validation for ExternalFunction, will validate other options
in follow on PRs.
This PR adds a feature flag to model a potential new-in-practice rule in React:
that freezing a function expression also freezes its closed-over values,
transitively. For example, in the following code `data` is frozen when the
lambda that captures it is is passed to useEffect:
```javascript
const data = [];
// useEffect freezes its argument (the function expr), which transitively
freezes its captured value data
useEffect(() => {
foo(data);
}, [data]);
data.push(true); // ERROR: mutating a frozen value
mutate(data); // we conservatively assume this doesn't mutate but could be wrong
```
Note that this rule has never been written down or enforced. It is theoretically
equivalent to the rule (already implemented in Forget) that values captured by
JSX are frozen:
```javascript
const style = {...};
<div style={style}>...</div>
style.width = 10; // ERROR: mutating a frozen value
mutate(style); // we conservatively assume this doesn't mutate but could be
wrong
```
However, JSX is typically constructed toward the very end of a render function.
Thus in practice there isn't much subsequent code that could even modify such a
captured value. But for the useEffect case (and other hooks that take closures
as arguments), they tend to occur much earlier in a render function. There's
more code that can run later and still modify the captured values, without
causing issues in practice. The _practical_ rule today is that you can't modify
values captured by frozen lambdas _after the component returns_: it's fine in
practice to modify captured values between calling eg useEffect and returning
from render.
Thus this feature flag is fairly likely to break some percent of real product
code. I'm adding this so that we can experiment and see how unsafe it actually
is.
Adds an internal compiler assertion pass which checks that all the instructions
which are necessary for constructing a given scope correctly end up within the
corresponding ReactiveScopeBlock. All known cases where this can occur are fixed
earlier in the stack, but this assertion will help us catch any other cases we
haven't thought of. See docblock comment for more info.
## Test Plan
I manually reverted the fixes from the previous PRs while keeping the new
fixtures, and verified that this new assertion pass flags the fixtures as
invalid.
The previous changes mostly meant that we removed the label terminal and didn't
have instructions for the same scope split in a way that we couldn't merge. But
logicals were still causing a split because MergeConsecutiveScopes can't merge
the blocks in that case. Here we move PruneUnusedLabels earlier in the pipeline
to ensure that instructions from IIFEs have floated up to the parent block scope
level.
Fixes for the previous PR. What was happening is that our inference was
inferring the correct mutable ranges and reactive scopes, but the inlining
process left the instructions from the IIFEs inside a separate block, with a
'label' terminal preceding it. When we converted to ReactiveFunction this was
preserved as a ReactiveLabelTerminal, which meant that the first instruction for
the mutable range could be nested inside one LabelTerminal, while more would be
in a subsequent LabelTerminal. But we close blocks based on the block scope!
This meant that we'd have leftover instructions (in the second LabelTerminal)
that got left out of the block.
Furthermore, because inlining was happening after EnterSSA we weren't creating
phis correctly. This PR fixes a bunch of these issues, and a subsequent PR
handles the remaining cases:
* We move DropManualMemo and InlineIIFEs before EnterSSA. This means we lose the
ability to use type information, but we ensure that we create proper SSA ids and
phis for any reassignments within the IIFE
* We also update PruneUnusedLabels to not just remove the unused labels, but to
actually remove LabelTerminals that don't need them.
We construct invalid mutable ranges in these cases because the range starts
within a labeled block. We need to run merge consecutive scopes and EnterSSA
after inlining so that the code is lifted out of the labeled block to the
correct scope, and so that we create phis for reassignments within the IIFE.
This has caused issues for people when things like Babel cause issues. It's not
actionable and it crashes eslint. Just like the Babel plugin, the eslint plugin
should never throw. Instead, let's log the error so the data isn't lost.
This combines our scripts and makes it so we no longer need to create a separate
commit to add the forget-feedback/dist directory into the react-forget repo. I
originally did that so I could run tests here, but now that the external repo is
created and has its test suite hooked up to CI, this is now unnecessary friction
to run a sync
All of these warnings go away: ``` ➜ playground git:(remove-prettier) ✗ yarn
dev yarn run v1.22.19 $ NODE_ENV=development && next dev ready - started server
on 0.0.0.0:3000, url: http://localhost:3000 warn - You have enabled
experimental feature (appDir) in next.config.js. warn - Experimental features
are not covered by semver, and may cause unexpected or broken application
behavior. Use at your own risk. info - Thank you for testing `appDir` please
leave your feedback at https://nextjs.link/app-feedback
event - compiled client and server successfully in 964 ms (199 modules) wait -
compiling /page (client and server)... warn -
../../node_modules/prettier/index.js Critical dependency: the request of a
dependency is an expression
../../node_modules/prettier/index.js Critical dependency: require function is
used in a way in which dependencies cannot be statically extracted
../../node_modules/prettier/index.js Critical dependency: the request of a
dependency is an expression
../../node_modules/prettier/index.js Critical dependency: the request of a
dependency is an expression
../../node_modules/prettier/third-party.js Critical dependency: the request of a
dependency is an expression wait - compiling... warn -
../../node_modules/prettier/index.js Critical dependency: the request of a
dependency is an expression
../../node_modules/prettier/index.js Critical dependency: require function is
used in a way in which dependencies cannot be statically extracted
../../node_modules/prettier/index.js Critical dependency: the request of a
dependency is an expression
../../node_modules/prettier/index.js Critical dependency: the request of a
dependency is an expression
../../node_modules/prettier/third-party.js Critical dependency: the request of a
dependency is an expression ```
Standalone prettier is meant to be used in the browser:
https://prettier.io/docs/en/browser
I don't know if it's possible to write a test for this as I can't seem to get
the codegen to change.
For the following testcase: ``` function useFoo(setOne) { let x; let y; if
(setOne) { x = 1; y = 3; } else { x = 2; y = 5; }
return { x, y }; } ```
The LeaveSSA changes from: ``` .... bb1 (block): predecessor blocks: bb2 bb3
x$36:TPrimitive: phi(bb2: x$19, bb3: x$19) y$21[8:14]:TPrimitive: phi(bb2:
y$21, bb3: y$21)
... ```
to ``` ... bb1 (block): predecessor blocks: bb2 bb3 x$36:TPrimitive:
phi(bb2: x$19, bb3: x$19) y$38:TPrimitive: phi(bb2: y$21, bb3: y$21) ... ```
Notice how `y`'s reassignment got skipped previously.
Previously if any operand was reactive, we transferred that reactivity to other
operands that had a mutable effect (capture, conditionally mutate, mutate, or
store). But a value can be captured without ever being modified again. This PR
updates the logic to only transfer reactivity among operands that are actually
mutable at the given instruction, based on the mutable range. This is strictly
more precise.
This PR adds one remaining feature to InferReactivePlaces: tracking indirections
like LoadLocal, PropertyLoad, and similar. Consider something like:
```
// INPUT
x.push(reactiveValue);
// HIR
t0 = LoadLocal 'x'
t1 = PropertyLoad t0, 'push'
t2 = LoadLocal 'reactiveValue' // reactive
t3 = CallExpression mutate t0 . read t1 ( read t2 )
```
Because a reactive value (`t2`) flows into `t0`, we want to record t0 as
reactive as well. But that's just the temporary for `LoadLocal 'x'` - what's
really happening is that from this point, `x` is reactive.
InferReactiveIdentifiers tracked this, and now that logic is ported into
InferReactivePlaces as well. That lets us remove all the actual inference from
InferReactiveIdentifiers.
Updates `InferReactivePlaces` to infer control dependencies. We build on the
formal definition of control dependencies, which is that statement S2 is
control-dependent on statement S1 if S1 is in the post-dominance-frontier of S2.
Intuitively, if S1 decides whether S2 is reached or not, then S1 is a control
dependency of S2. The post dominance frontier of a given statement S is the set
of statements which may or may not reach S, and captures the intuitive notion.
We take advantage of phis: phis are the point where a variable may have multiple
values depending on the path we took. If a phi is not already known to be
reactive from data dependencies we check for control dependencies. Specifically
we look at each phi operand. We check if the block that the operand came from
has any reactive control dependencies, and if so we mark the phi itself as
reactive.
The post-dominance-frontier (PDF) algorithm requires walking the post-dominator
tree a bunch, so we cache the PDF of blocks so that we don't have to recalculate
on subsequent iterations.
In addition, `InferReactiveIdentifiers` now uses the _union_ of its own
inference plus the new `InferReactivePlaces` output when deciding what
identifiers are reactive. This ensures that control dependencies are recorded
correctly, fixing the previous test cases. The next diff adds the remaining
features to InferReactivePlaces so that it can fully replace
InferReactiveIdentifiers.
See context from #2187 for background about control dependencies.
Our current `PruneNonReactiveIdentifiers` pass runs on ReactiveFunction, after
scope construction, and removes scope dependencies that aren't reactive. It
works by first building up a set of reactive identifiers in
`InferReactiveIdentifiers`, then walking the ReactiveFunction and pruning any
scope dependencies that aren't in that set.
The challenge is control variables, as demonstrated by the test cases in #2184.
`InferReactiveIdentifiers` runs against ReactiveFunction, and when we initially
wrote it we didn't consider control variables. To handle control variables we
really need to use precise control- & data-flow analysis, which is much easier
with HIR.
This PR adds the start of `InferReactivePlaces`, which annotates each `Place`
with whether it is reactive or not. This allows the annotation to survive
LeaveSSA, which swaps out the identifiers of places but leaves other properties
as-is. This version does _not_ yet handle control variables, but it's already
more precise than our existing inference. In our current inference, if `x` is
ever assigned a reactive value, then all `x`s are marked reactive. In our new
inference, each instance of `x` (each Place) gets a separate flag based on
whether x can actually be reactive at that point in the program.
There are two main next steps (in follow-up PRs):
* Update the mechanism by which we prune non-reactive dependencies from scopes.
* Handle control variables. I think we may be able to use dominator trees to
figure out the set of basic blocks whose reachability is gated by the control
variables. This should clearly work for if/else and switch, as for loops i'm not
sure but intuitively it seems right.
This is part of a stack for inferring variables which are reactive via *control
dependencies* as opposed to a data dependency. In compiler engineering, a
statement S2 is control-dependent on statement S1 if S1 is in the post-dominance
frontier of S2. Stated more intuitively: if S1 decides whether or not S2 is
reached, then S1 is a control dependency of S2.
As a start, we add `Place.reactive: boolean` so that individual places can track
whether they are reactive or not. This lets us do fine-grained reactivity
inference on the control-flow graph, even taking into account different SSA
instances of a variable, so that we can say that a particular SSA version of `x`
is reactive, while other "versions" of x (due to reassignment) are not.
Adds a toy next.js app which doesn't do anything interesting. It has a single
e2e test which can be run via `npm run test:e2e`, which checks if a `$` variable
was injected by the compiler, as a sanity check whenever we commit a new version
of the compiler to the external repo.
Not every consumer of Forget will be able to run an experimental version of
React. In the meantime before useMemoCache is stable, provide a way for OSS to
pass in a userspace impl.
Turns out we were only using this in PrintHIR and in the logger, so it should be
safe to inline and makes installing the plugin for external collaborators a
little easier.
I'm assuming inlining this is ok because we only need to defer to the host's
Babel version(s) when it affects parsing or the final codegen output, hence why
we continue to no-inline @babel/types – as it is responsible for creating AST
nodes during codegen
The flattening broke the shell script because the directory structure changed.
Instead of depending on a flaky shell script, this PR encodes the commands as
part of the github workflow.
Attempt to bundle the plugin with rollup instead of just tsc. I'm intentionally
not inlining babel related modules, as we should ideally rely on the host
environment's version instead
We can now fully remove prettier from babel-plugin-react-forget. I moved it to
devDependencies instead, to make the rollup build simpler and so we can continue
to prettify our internal source code.
This moves prettier formatting into fixture-test-utils instead, so we can remove
the dependency on prettier in the babel plugin. I want to do this because I
don't want to include prettier in the rolledup artifact when we build the babel
plugin.
Runs Forget on the playground, so we can dogfood Forget and experiment with
improvements to UX.
Opts out of compiling the layout page because thats run on the server which
doesn't seem to have access to useMemoCache.
This kept throwing warnings in our playground build because prettier uses node
APIs and is not meant to be bundled and run on the browser.
There's a separate standalone version that runs on the browser. A follow on PR
will make the playground use that but this PR is a first step towards using a
standalone prettier.
Currently DCE can remove variable declarations that are unused, ie where all
control-flow paths to usage of the variable are overwritten by a reassignment.
We then have to reconstruct the original variable declaration at the appropriate
block scope during LeaveSSA, which is complex and can actually be incorrect in
some cases.
This PR updates to ensure that DCE will not remove the original variable
declaration for any variable that is used (even in the case of always being
reassigned before use). The main changes are:
* DCE retains variable declarations, but if a variable declaration is always
shadowed by reassignments then DCE will rewrite StoreLocal -> DeclareLocal so
that it can DCE the unused initial value.
* BuildHIR now has to change its handling for reassignment destructure
instructions with nesting. Nesting uses a temporary which would appear as a
declaration of a new variable, which is incompatible with other reassignments.
See comments in the file.
* LeaveSSA is quite a bit simpler now, since we never need to reconstruct a
declaration.
Adds test cases for all the cases of control dependencies that I can think of.
We don't currently handle control dependencies correctly in any of these cases.
There's also another test case which demonstrates why reactive dependency
inference needs to be fixpoint, even for non-control dependencies.
Reactive scopes are currently preceded by individual variable declarations, one
for each of the scope's dependencies. Only after checking independently if each
of these dependencies has changed do we then do an "or" to check if we should
actually recompute the body of the scope.
But that's wasteful: it's more efficient to rely on `||` short-circuiting, and
recompute as soon as any input changes.
The main reason I can see not to do this is debugging. Having the change
variables makes it easy to see what changed. But at this point I think it makes
sense to optimize for code size and performance; we can always add back a
dev-only mode that uses change variables if that turns out to help debugging.
Rewrites the core logic of MergeConsecutiveScopes to be easier to follow and fix
bugs. We now do a two-pass approach:
* First we iterate block instructions to identify scopes which can be merged,
without actually merging the instructions themselves.
* Then we iterate again, copying instructions from the block either into the new
output block, or into their merged scope, as appropriate.
I think the simplicity here is worth the performance cost, and we can always
revisit later as necessary.
This morning @mofeiZ reminded me that our codegen doesn't really have any guards
against reordering, since temporaries are lazily emitted. We're relying on the
fact that our lowering and memoization carefully preserves order of evaluation,
such that delaying the instructions in codegen doesn't change semantics.
To help catch any mistakes with this, I had previously added code that reset the
codegen context's temporaries before/after exiting a reactive scope. That
ensured that temporaries from within the scope weren't accessible outside it.
This PR extends that approach to _all_ blocks, so that temporaries created
within a block aren't accessible outside it.
I'm also going to explore more actively resetting temporaries after they
"should" be used. There are a couple cases where temporaries are reused, though,
which we have to change first.
So far we've been preserving JSX whitespace all the way through to codegen. But
JSX has clear rules around whitespace handling, which allows us to trim
whitespace in the input in lots of cases. For the most part this doesn't change
our output, but I think that’s generally because of prettier. This PR should
make a big difference when debugging the compiler, by removing all the
whitespace JsxText values.
But in some edge-cases it really makes a difference in the output since we can
avoid memo slots for strings like `"\n "`..
## Test Plan
* Experimented with our internal tool to verify transform output to confirm that
JSXText whitespace does not impact fbt transform results.
* Synced and tested profile page, looks fine
Rewrites the core logic of MergeConsecutiveScopes to be easier to follow and fix
bugs. We now do a two-pass approach:
* First we iterate block instructions to identify scopes which can be merged,
without actually merging the instructions themselves.
* Then we iterate again, copying instructions from the block either into the new
output block, or into their merged scope, as appropriate.
I think the simplicity here is worth the performance cost, and we can always
revisit later as necessary.
This is a repro for a bug that occurs when `enableMergeConsecutiveScopes` is
enabled. Reminder that this feature is off by default and not enabled yet
internally.
I found this via #2121, where eliminating extraneous whitespace JSXText
instructions meant that MergeConsecutiveScopes started merging a fixture
differently, revealing a bug. This PR reproduces that case by keeping the
identical structure, but using plain objects to represent the JSX elements
instead of JSX syntax.
This PR completes the refactor. We now do the following sequence:
* ValidateUseMemo. This is a new pass that extracts just the validation logic
from the existing InlineUseMemo. This was always being run before, so this pass
also always runs.
* DropManualMemoization. As before, this converts useMemo calls into an IIFE
(immediately invoked function expression).
* InlineImmediatelyInvokedFunctionExpressions (prev InlineUseMemo). This pass
now inlines _all_ IIFEs, including both useMemo calls that were dropped as well
as IIFEs that the user wrote.
The motivation for this change is that some codebases use IIFEs as a workaround
for lack of if expressions, but we're unable to optimize within function
expressions. This is the reason we originally added inlining for useMemo, but
given that IIFEs are common it makes sense to generalize the inlining.
## Test Plan
* Manually checked changes in output
* Synced internally and tested on profile page, no issues observed. Also
spot-checked some of the changes in ouput and it looks as expected.
The goal of this stack is to generalize `InlineUseMemo` into a pass that inlines
all immediately invoked function expressions (IIFEs). Rather than specialize
just useMemo calls, we'll rely on DropManualMemoization running first and
turning useMemo calls into IIFEs. Then the generalized inlining pass can handle
those IIFEs as well as others present in the source.
For now, moving the order of the pass makes the output closer to what it will
eventually be after this stack is complete.
#2127 introduced a special type for the result of `useContext()` that was sort
of ref-like. The intent was to allow code like this:
```
function Foo() {
const cx = useContext(...);
function onEvent() {
cx.foo = true;
};
return <Bar onEvent={onEvent} />;
}
```
However, that code actually is allowed by the compiler by default. It's only a
bailout when `@validateFrozenLambdas` is enabled. The "fix" in #2127 therefore
wasn't strictly necessary to unblock rollout, and it's also flawed in a few
ways:
* First, `useContext(FooContext)` should have equivalent behavior to a custom
hooks which does the same thing, ie `function useFooContext() { return
useContext(FooContext) }`. Specializing the type of useContext makes the
behavior different.
* Second, it meant that even readonly accesses of the context inside a callback
marked the function as capturing, which in turn prevented those callbacks from
being memoized.
So i'm reverting this and we'll have to think a bit more about this case.
---
Patch for original patch of injecting `useMemoCache` logic 😅 This is failing on
a good number React components internally (not in our current experiments)
The playground now uses the new pragma parser so it's guaranteed to use the
right defaults and have consistent parsing with snap/sprout. In addition, we now
emit a debug event from the compiler which contains pretty-printed environment
config, making it easy to check which settings are being applied in playground.
<img width="3008" alt="Screenshot 2023-10-05 at 11 05 58 AM"
src="https://github.com/facebook/react-forget/assets/6425824/2417f40c-1320-4c39-a661-a4e34e3d69c4">
Updates Snap and Sprout to use the new pragma parser, which also means they will
always use the same default flags as the compiler itself sets. A side benefit of
this is that you no longer need to rebuild snap/sprout to update their flags,
since they will take flags from the version of the compiler being executed.
Adds a helper function for parsing pragma strings to the compiler itself, and
exports it. This will be used in follow-ups to make Snap, Sprout, and Playground
all use the same pragma parser. The helper also starts from the default values,
so adopting this will also make it easy for all those places to have the same
defaults automatically.
Adds support for empty catch clauses in a try/catch. We add a new block kind
`catch` which prevents the empty catch block from being merged with other types
of blocks, preserving the block structure within the HIR and allowing us to
reconstruct the empty catch.
---
Implements popular feature request ✨ per feedback from a majority of snap users.
**Add `@debug` to the first line of your `testfilter.txt` file to opt into
implicit debug mode**, in which debug logging is enabled anytime filter mode is
on + only one fixture file is found.
- live edits to testfilter.txt are reflected in watch mode, so you can add /
remove `@debug` to `testfilter.txt` in the middle of a watch session
- I personally don't use debug mode all the time (I often a single file filtered
+ a lot of console log traces), but it should be easy to add `@debug` to the top
of `testfilter.txt` and leave it there forever.
This is a (hopefully) better approach at printing ReactiveFunction. The nesting
wasn't always clear in the previous version, this should help. See playground to
experiment.
Updates `Environment` to store all feature flags on a single `config` object. We
now also define an object with all the default config values, and use this to
populate defaults for any missing values in the user-provided config.
Most of our feature flags are accessed via the `Environment`, but a few cases
have slipped in where we look at the `config` object directly. The problem is
that the config object doesn't set defaults, so the check is effectively
encoding what the default is.
This PR moves to always accessing flags off of the environment, and adds a few
flags that weren't yet defined there.
Previously, we stored a global count variable that was updated every time we
added a property to the `arg` object. This was added to prevent collisions, and
make sure we do actually mutate the object.
But the count value was shared by the forget compiled and uncompiled versions,
so the same object mutated in either versions would result in having different
properties leading to potential test failures.
Instead, let's make count local and attempt to incrementally mutate the object
with different keys.
InferReferenceEffects uses object identity to merge states, which breaks when we
create a new object to model `undefined`.
Two value objects representing `undefined` are not equal due to referential
equality.
Instead, let's use a singleton to represent `undefined` value.
Handles an edge-case from earlier in the stack. When looking up a property on a
shape, if the property is defined we return it. But if it isn't defined, and the
property name is a hook, we treat it like a default custom hook.
Allows using hooks/methods off of the `React` namespace, for example
`React.useState(sathya)`. Thanks to the previous PR we correctly handle things
like validation of hooks called via propertyload syntax. The main change here is
to teach the compiler about the `React` namespace. This is a bit of a hack since
we treat it as a global, but we're transforming React code so this seems
reasonable (?).
There are a few additional touch-ups which I'll do in subsequent PRs to make
review easier. For example, we need to teach our useMemo/useCallback flattening
logic to also handle the case of `React.useMemo()` etc.
Hooks can be called via method call syntax, eg `Foo.useBar(sathya)`. This PR
teaches the compiler about this form of hooks for things like flattening scopes
with hooks, validating conditional hooks, etc.
Note that we still disallow calls on the React namespace, so things like
`React.useState()` continue to error. That's the next PR in the stack!
Adds a new type for representing context values, which is transitive. So
`useContext(a).b.c` also gets inferred as a context type. This allows us to
refine our inference, and allow passing callbacks that modify context where a
"frozen" lambda is exepcted.
This is a distilled version of the duplicate declaration @mofeiZ and I saw when
trying to sync latest Forget internally, plus a fix to avoid the duplicate
instruction.
Fixes an issue with incorrect spacing where spaces were getting dropped, despite
an explicit `{" "}` in the input. The issue is that we didn't maintain JSXText
all the way through compilation. BuildHIR distinguishes string literals (such as
the above, inside an expressioncontainer) from JSXText, and we propagate this
distinction all the way through to codegen.
But then codegen stores temporary values as `t.Expression` nodes, which means we
have to convert the JSXText nodes to StringLiteral and we lose the distinction.
This PR updates codegen to save temporaries as `t.Expression | t.JSXText` so
that we can preserve the difference. In most places we just coerce the value to
an expression, but the code for emitting JSX child items looks at the raw value
so it can distinguish them. JSXText is emitted as-is, while StringLiterals are
always wrapped in an expression container.
See the new test case which demonstrates the expression being preserved.
Object methods must not be cached independently, so this PR flattens the
reactive scope to prevent memoization.
In the future, we can combine the FlattenScopesWithObjectMethods and
FlattedScopesWithHooks passes by making them more modular. But this works for
now.
Object methods are lowered to functions and added to ObjectExpression. The
codegen is interesting because we shouldn't emit code that lowers the object
method into a separate statement and then stores it into an object expression.
An shorthand object method has different semantics than an object method using
the function syntax, so we need to preserve the shorthand object method syntax
in the generated code.
To do this, we don't immediately generate an AST node for the ObjectMethod but
instead store it in a side table during codegen. Only when emitting code for an
ObjectExpression, we lookup this side table and emit the object method inline in
the body.
Adds support for lowering rest element parameters to spreads. We eagerly create
a temporary, similar to the approach for destructuring. In theory we could do
something more optimal if you have a `...foo` (rest element where the argument
is an Identifier) but it doesn't seem worth optimizing yet.
As noted earlier in the stack and in chat, there are some cases where the output
of a reactive scope is not guaranteed to change just because its inputs did.
Consider a function `foo(x) { return x < 10 }`. If x was 0 and changes to 1, the
result of `foo(x)` won't change.
For code such as `[foo(x)]`, then, merging the scope for `t0 = foo(x)` and `t1 =
[t0]` into a single `t1 = [foo(x)]` could cause us to invalidate `t1`
unnecessarily. For example, x changing from 0 to 1 would allocate a new array of
`[true]` even though the value didn't meaningfully change. This is the second
category of merging, where we merge scopes A and B if the outputs of A are the
inputs to B.
With this change, we only do this type of merge if the outputs of the first
scope are known to invalidate whenever the input does. We're conservative about
this, and only consider function expressions, arrays, object, and jsx to always
invalidate. Function calls _may_ invalidate, but as w the `foo()` example here
they also may not.
Note that this is purely about optimization and not correctness. We could always
merge in this case (per earlier in the stack) but that might invalidate more
often than we would like.
This is the optimization mentioned in #2113. When we merge scopes, often the
declarations from the first scope become unnecessary, since those values are
only consumed by the subsequent, now-merged scope. There's no point emitting
those values as outputs of the merged scope since no one can consume them.
Thanks to the logic in #2116 we now know the last place each identifier is used.
We use that again here, to prune declarations that aren't used past the end of
the merged scope. This is a pretty dramatic win on cache slots used.
We can't merge scopes if the intervening instructions produce values that are
used later on. This PR improves the mechanism for detecting this case: first we
build up a mapping of the last time (max instruction id) each identifier is
used. Then when we're about to merge scopes we check if all the intervening
lvalues are last used at or before the scope. If so that means it's safe to
merge.
I can't think of any edge cases that are problematic in the old behavior, but
this version is more trivially correct and should allow us to extend to other
types of instructions more easily.
This is something we've wanted to do for a while, and which @sophiebits also
brought up. The idea is to merge consecutive reactive scopes that will always
invalidate together. There are two cases of this:
* Both scopes have the exact same inputs
* Or the inputs of the second scope are the outputs of the first
In both these cases it's pure overhead to keep the scopes separate. In the first
case where both scopes have the same inputs, merging allows us to check the
inputs once instead of 2+ times. In the second case, we know that the second
scope will invalidate when the first does so it's wasteful to recheck the
outputs of the first scope for changes.
This is already cutting down on memoization quite a bit in the fixtures. Note
that there's an additional optimization we can do after merging the second
category, which is to remove the first scope's declarations if they were only
referenced by the second scope. I'll add that in a follow-up.
---
(pasted from comment)
Instead of handling holey arrays, bail out with a TODO error.
Older versions of babel seem to have inconsistent handling of holey arrays, at
least when paired with HermesParser. When using these versions, we should bail
out instead of throwing a Babel validation error.
Issue:
The babel ast definition for array elements changed from Array<PatternLike> to
Array<PatternLike | null>. Older versions do not expect null in the ArrayPattern
ast and will throw a validation error during Codegen.
- HermesParser will parse [, b] into [NodePath<null>, NodePath<Identifier>]
- Forget will try to preserve this holey array when we codegen back to js
(e.g. we call a babel builder function arrayPattern([null, identifier]))
- Babel will fail with `TypeError: Property elements[0] of ArrayPattern
expected node to be of a type ["PatternLike"] but instead got null`
PR that changed the AST definition:
https://github.com/babel/babel/pull/10917/files#diff-19b555d2f3904c206af406540d9df200b1e16befedb83ff39ebfcbd876f7fa8aL52-R56
[ez] Add TODO bailout on non-backward compatible holey arrays
---
(pasted from comment)
Instead of handling holey arrays, bail out with a TODO error.
Older versions of babel seem to have inconsistent handling of holey arrays, at
least when paired with HermesParser. When using these versions, we should bail
out instead of throwing a Babel validation error.
Issue:
The babel ast definition for array elements changed from Array<PatternLike> to
Array<PatternLike | null>. Older versions do not expect null in the ArrayPattern
ast and will throw a validation error during Codegen.
- HermesParser will parse [, b] into [NodePath<null>, NodePath<Identifier>]
- Forget will try to preserve this holey array when we codegen back to js
(e.g. we call a babel builder function arrayPattern([null, identifier]))
- Babel will fail with `TypeError: Property elements[0] of ArrayPattern
expected node to be of a type ["PatternLike"] but instead got null`
PR that changed the AST definition:
https://github.com/babel/babel/pull/10917/files#diff-19b555d2f3904c206af406540d9df200b1e16befedb83ff39ebfcbd876f7fa8aL52-R56
This PR adds preliminary support for hoisting const variable declarations. We do
this via BuildHIR when lowering top level statements in a BlockStatement, by
first checking which bindings are in scope to be hoistable if referenced before
they are declared. The declarations are then hoisted to their earliest point
where they are referenced (ie the top level statement just before) as context
variables.
Later, prior to codegen, we restore the original source by removing the
DeclareContexts and transforming their associated StoreContexts back.
Support for hoisting other kinds of declarations will come in future PRs!
This is a redo of #1640 now that we've established the necessary infrastructure,
most notably `Effect.ConditionallyMutate` and `noAlias` from #2103 earlier in
this stack. We can now understand the semantics of hooks that return deeply
readonly values composed of primitives, arrays, or objects such that any
`.map()` or `.filter()` calls are guaranteed to be the corresponding array
methods. That further allows us to refine, since we know that the lambdas passed
to these calls can't alias, are conditionally mutable, etc. All in all this
should let us memoize less in practice.
Adds `noAlias` support for CallExpression, including hooks. Note that we treat
hook arguments as escaping by default — ie we assume that they don't just flow
into the hook return value, but are just outright escape points equivalent to a
return. A `noAlias` annotation on a hook definition disables both: this will
allow us to avoid memoizing the `graphql` tag arguments to `useFragment`, for
example.
Skips compilation of code that has a reference to `useMemoCache()`, as a
last-resort to avoid double-compilation of code. This is meant as a quick way to
unblock since we're still seeing some double compilation issues when syncing
internally.
This has been nagging at me for a _long_ time: we unnecessarily memoize function
callbacks passed to things like Array.prototype.map, even though we know these
functions can't escape. This PR fixes this as follows:
* Adds a `noAlias?: boolean` flag to builtin function signatures, defaulting to
false if not specified.
* Adds a feature flag, `enableNoAliasOptimizations`, to gate optimizations based
on the value of that new flag.
* When the feature is enabled, `PruneNonEscapingScopes` now looks up the
signature of method calls, and avoids memoizing the arguments if the signatures
specifies `noAlias: true`.
* Annotates Array.prototype.map and Array.prototype.filter as `noAlias`.
This does not mean we'll never memoize arguments to Array.prototype.map, it just
means that the argument itself won't be considered as escaping. If the function
still escapes by some other means it will get memoized:
```
function Component(props) {
const f = () => {}; // memoized!
const x = [].map(f); // not from here..
return [x, f]; // but bc it escapes here
}
```
Note: this delivers some of the wins from #1640. That PR tried to do a bunch of
things, part of which I already landed w the introduction of
ConditionallyMutate, which allowed us to type Array.prototype.map. This PR
further gives us the ability to understand functions that don't alias their
params at all. The remaining bit from #1640 is the idea of understanding that
key hooks such as `useFragment()` return transitively readonly, transitively
array/object/primitive values, and any `.map()` or `.filter()` calls must be on
arrays, allowing us to optimize them. Without that extra step, we'll still have
to memoize a lot of `array.map()` lambdas just because we aren't sure that the
receiver is an Array. But this PR helps with some cases, and lays the groundwork
for the rest of that PR.
Per the title, `<fbt:param>0</fbt:param>` is invalid FBT, you must wrap the text
in an expression container. But that's not all, `fbt:param` can only have a
single child, which means we have to strip out the text elements that occur from
the whitespace in the source.
Fixes another special-case rule of fbt that i wasn't aware of: apparently
`<fbt:param>` elements don't have to appear as direct children of `<fbt>`, they
can be nested, and in this case they must appear as direct children of the fbt
and not via an identifier indirection. This PR recursively extends the scope of
FBT operands to make this work.
The type of ObjectProperty is specific to the key, not the ObjectProperty. In
the future, we want to add a type to represent the value of the ObjectProperty.
This PR moves out the fields related to the key of the ObjectProperty to a
separate type.
---
Recent commits added calls to `NodePath.hasNode`, which does not exist in babel
v7.1.6 (internal). Forget is failing with this error.
```js
TypeError: handlerPath.hasNode is not a function
at lowerStatement (.../babel-plugin-react-forget/HIR/BuildHIR.js:924:58)
...
```
@mofeiZ noticed that some configurations of prettier don't support JSXFragment
appearing as a JSXAttribute value, in violation of the spec. Coincidentally I
noticed that our internal build system also fails on this as i was trying to
roll out on more surfaces. This PR makes sure we wrap fragments in an
expressioncontainer if they appear as jsxattribute values.
Adds test cases that would demonstrate different output if we were to update
InferTypes to infer the types of phi identifiers when their operands have the
same type. We currently do infer such types, but attach them to phi.type instead
of phi.id.type, so the type doesn't effect inference. Fixing that would cause
the output on these examples to change — however, per the discussion on #2079,
we'd also incorrectly set the mutable ranges in some cases and cause incorrect
compilation.
I did a thorough review of InferReferenceEffects and can't figure out how to
trigger the bug in our current implementation — the bug only kicks in when phi
operands would be mutated as a store instead of a mutate, and that cannot happen
given our currently imprecise types on phi identifiers.
---
Changed `panicOnBailout: boolean` to `panicThreshold`, which has the following
options. Note that `ALL_ERRORS` corresponds to `panicOnBailout = true` and
`CRITICAL_ERRORS` corresponds to `panicOnBailout = false`. `NONE` is a new
option.
```js
export type PanicThresholdOptions =
// Bail out of compilation on all errors by throwing an exception.
| "ALL_ERRORS"
// Bail out of compilation only on critical or unrecognized errors.
// Instead, silently skip the erroring function.
| "CRITICAL_ERRORS"
// Never bail out of compilation. Instead, silently skip the erroring
// function or file.
| "NONE";
```
Jest seems to run babel through a different pipeline than Metro and -
(perhaps through its complex `require` interjection logic). When running jest
tests, exceptions thrown by babel transforms will bubble up to the nearest
exception boundary which is often the jest test itself. This may not be a useful
signal to anyone running a jest test with Forget enabled, as the erroring code
may be within Forget itself or a transitively required module.
Another reason to immediately bailing out on critical errors is that we may want
to record errors found in the rest of the file.
---
I'm not convinced that this change makes sense. A counterargument is that any
CriticalErrors *should* be reported as parse errors, regardless of the runtime
mode. Anyhow, this would be useful long term for our static analysis scripts
(e.g. collecting info on bailouts and compilation info e.g. # slots used for
JSX) as we want to compile-as-much-as-possible in those.
---
Add types for logged events, including a `CompileSuccess` event which can help
us record successfully compiled Forget functions and their compilation details
(e.g. # memoSlots used).
We currently have multiple flags for targeting which functions to compile, but
they are actually mutually exclusive. This PR consolidates to a single
`compilationMode: 'annotation' | 'infer' | 'all'` flag:
* Annotation compiles only functions that explicitly opt-in with "use forget"
* Infer compiles explicitly opted-in functions (via "use forget") as well as any
known/inferred components or hooks:
* Component declarations
* Component or hook-like functions (same rules as the ESLint plugin but with an
extra check for whether it uses JSX or calls a hook)
* All compiles all top-level functions. We should get rid of this in a follow-up
and make tests use infer mode by default, and add explicit opt-ins where
necessary.
In all modes, "use no forget" always takes precedence and can be used to
opt-out. The default mode is now "infer".
Replaces the use of `NextIterableOf` in for-in with a new `NextPropertyOf`
instruction. The key distinction is `for-of` invokes an arbitrary iterator,
which means a) each iteration may mutate the collection being iterated and b)
the returned value may be mutable. However, `for-in` invokes a language-level
mechanism to iterate: simply iterating alone _cannot_ modify the collection, and
the returned value is known to be a primitive.
First pass of ForInStatement support. This mostly just copies our handling of
ForOfStatement, but the next PR updates to use a different instruction instead
of `NextIterableOf`.
Supports call expressions if the callee and args are themselves reorderable. As
part of this I realized that we currently allow identifier references to be
reordered. To be safe, this PR updates the logic to continue consider
identifiers as reorderable, but considers an arrow function to be not
reorderable if it contains a local variable reference. We can likely relax that
rule, but this quickly unblocks the next experiment.
Supports reordering of unary and arrow function expressions:
* Supports a trivially safe subset of unary operators, rejects things like
`void` just because we don't need it yet.
* Supports arrow function expressions that are either an empty block statement
or a single expression which is itself reorderable.
Adds handling for some cases where the handler is unreachable (or is provably
unreachable after analysis & optimization), where the try/catch can be flattened
away:
* The try block is empty. Nothing can throw, so the handler is unreachable.
* The try block will always return. It can't return anything interesting (ie the
result of a function call or variable load) since those could throw, but a try
block that always return a primitive means the handler is unreachable.
* The same, except where we only determine that the try block always returns via
constant propagation.
Enables sprout for all the new try/catch fixtures in this stack. I added new
helpers and tried to make sure we're testing the most interesting codepath of
each fixture. This is where property testing would help, since we could test
multiple paths with a single block of code, but for now this seems like a good
balance of coverage.
It's possible that the value thrown during a `try` block actually is a reference
to some value defined outside the scope of the try block. If the catch clause
param is also mutated, that means the mutable range of the variable would have
to include the entire try/catch.
We handle this by emitting a DeclareLocal temporary for the catch param prior to
the try/catch. If it is modified during the catch block, that will extend its
mutable range to cover the full try/catch. If any values are mutated inside the
try, their range will also (naturally) extend around the full try/catch block.
These ranges will overlap and be merged, ensuring that we capture the
possibility that the value is mutated via the catch param. See unit test.
Modeling `throw` inside of a try/catch is awkward because it's basically a
variable reassignment and a goto together. Thankfully that is an antipattern —
using exceptions instead of control-flow — so it seems pretty reasonable to just
put a todo here and leave it.
Adds an optimization pass to prune unnecessary maybe-throw terminals, when the
block can be proven not to throw. For now we're _very_ conservative about what
instructions we consider not to throw. There isn't too much of an advantage in
pruning further, either.
This PR also updates BuildReactiveFunction to handle the possibility of early
returns within try or catch blocks, making sure we don't hit the invariant of
emitting the same block twice.
Implements the core lowering logic for try/catch:
* Inside of the `try` block, we use the new HIRBuilder mode to wrap every
instruction in a separate basic block with a maybe-throw terminal
* We emit a 'try' terminal for the try/catch itself
For basic examples this already works correctly. But we don't handle catch
clause params yet.
This PR adds the other piece, a 'try' terminal which represents try/catch and
the possibility of fallthrough to the code afterwards. For now `finally` is
unsupported. We don't yet produce these terminals, see later in the stack.
Adds a "maybe-throw" terminal which represents the possibility that the block
may or may not throw, and can either continue forward or exit to an exception
handler (`catch`). Also updates HIRBuilder to track the current mode, and when
inside a try block to wrap every instruction inside a basic block that ends in a
maybe-throw.
So far this code isn't used yet, so doesn't affect output.
Mostly reuses existing analysis of an identifier property key.
Adds a new type field to ObjectProperty to propogate the type of the key. This
is used in codegen to correctly emit a string literal or an identifier.
Technically there is a body node created for implicit return expression in a
arrow function, so the existing logic should've worked fine. But there seems to
be a Babel bug, so let's work around it by traversing the function.
Added a test case that captures a dep as a param -- this is currently
unsupported and also something that would've been ignored before this PR. Added
a failing test to make sure we think about this case when we add support for
default params.
Future passes will lead to inconsistent state between passes and will require
passes to run in a specific order, so let's make sure no one will misconfigure
the passes.
Internal version of babel/core doesn't support the `inherits` property, so let's
try removing it. The tests still seem to pass, so this might be vestigial from
the first iteration of the compiler from last year
Instead of emitting a memo block, emit a function expression and pass it as an
argument to derived (which will then create a computed).
The naming of 'derived' needs to be tweaked.
Minimal repro extracted from our internal codebase. Our inference mode sees that
this arrow function is component-like and attempts to compile it, which then
fails because the function accesses `this` which we bailout on.
---
quality of life improvement as this seemed to be confusing (oops, and thanks for
the feedback!)
We don't *really* need static annotations for whether a function returns jsx
(e.g. should be rendered as a React element) or not (e.g. should be wrapped in a
wrapper component. This PR adds check for returned jsx objects at runtime
---
Tested by running diffing the output of `yarn sprout --verbose` between this PR
and base.
I ran into the same issue that @poteto and @gsathya (and probably @mofeiZ) have
run into: "Duplicate declaration of '$'" caused by Babel visiting a function
twice despite our calling `skip()`. This PR keeps a set of nodes that we have
already visited to avoid visiting them again, as a workaround for skip not
working.
# Test Plan
Synced to www and confirmed that the previous bug no longer reproduces, and the
compiled output looks sane.
Completes a todo (ie fixes a silly mistake) from a PR earlier in the stack, so
we now correctly recognize and compile arguments to `React.forwardRef()` and
`React.memo()`.
This PR changes the way we compile ArrowFunctionExpression to allow compiling
more cases, such as within `React.forwardRef()`. We no longer convert arrow
functions to function declarations. Instead, CodeGenerator emits a generic
`CodegenFunction` type, and `Program.ts` is responsible for converting that to
the appropriate type. The rule is basically:
* Retain the original node type by default. Function declaration in, function
declaration out. Arrow function in, arrow function out.
* When gating is enabled, we emit a ConditionalExpression instead of creating a
temporary variable. If the original (and hence compiled) functions are function
declarations, we force them into FunctionExpressions only here, since we need an
expression for each branch of the conditional. Then the rules are:
* If this is a `export function Foo` ie a named export, replace it with a
variable declaration with the conditional expression as the initializer, and the
function name as the variable name.
* Else, just replace the original function node with the conditional. This works
for all other cases.
I'm open to feedback but this seems like a pretty robust approach and will allow
us to support a lot of real-world cases that we didn't yet, so i think we need
_something_ in this direction.
We currently have multiple flags for targeting which functions to compile, but
they are actually mutually exclusive. This PR consolidates to a single
`compilationMode: 'annotation' | 'infer' | 'all'` flag:
* Annotation compiles only functions that explicitly opt-in with "use forget"
* Infer compiles explicitly opted-in functions (via "use forget") as well as any
known/inferred components or hooks:
* Component declarations
* Component or hook-like functions (same rules as the ESLint plugin but with an
extra check for whether it uses JSX or calls a hook)
* All compiles all top-level functions. We should get rid of this in a follow-up
and make tests use infer mode by default, and add explicit opt-ins where
necessary.
In all modes, "use no forget" always takes precedence and can be used to
opt-out. The default mode is now "infer".
Adds a new option to infer which functions to compile, based on React's ESLint
rule. The main difference is that in addition to checking the function name we
also check that it creates JSX or calls a hook. This should cover a significant
majority of components and reduce the chance of accidentally targeting
non-components, but it will leave some false negatives.
Note that some cases that the ESLint plugin infers as React functions don't work
yet: we don't compile FunctionExpressions, only ArrowFunctionExpressions, and
the way we handle ArrowFunctionExpression doesn't work with things like
forwardRef or variable declarations. We'll need more updates to fully handle all
these cases, which I'll do later in the stack.
Updates the parser-benchmark script to use a benchmarking framework, and
also brings in yargs to make it easier to handle parsing arguments.
Non-CI usage:
```
$ cd bench/parser-benchmark
$ yarn bench --help
Options:
--help Show help [boolean]
--ci [boolean] [default: false]
--sampleSize [number] [default: 1]
$ yarn bench --sampleSize=3
[ISOBENCH ENDED] Parser Benchmark (3 times)
OXC - 32 op/s. 3 samples in 9518 ms. 5.288x (BEST)
SWC - 20 op/s. 3 samples in 9487 ms. 3.256x
HermesParser - 6 op/s. 3 samples in 9124 ms. 1.000x (WORST)
Forget (Rust) - 15 op/s. 3 samples in 9749 ms. 2.499x
```
In CI this outputs JSON instead of logging to stdout. The results are
stored in github's action cache and used as a point of comparison when
new commits are pushed to main. The comparison should be shown on
workflow [summary
pages](https://github.com/facebook/react-forget/actions/runs/5956421081), but
nothing will be populated until we merge this PR.
We intentionally only run this workflow on main as any run would
override the last cached result, so a PR with multiple pushes would then
start having comparisons with itself.
Sorry about the thrash in advance! This removes the top level `forget` directory
which adds unnecessary nesting to our repo
Hopefully everything still works
ESTree expects the `range` value to be an array of `[start, end]`, we support
deserializing from that format but serialized as an object of `{start,end}`, now
we serialize to the array form.
We’ve had this feature turned on internally for a while with only a couple minor
bugs and no fundamental issues. It’s a necessary change for simplifying function
expression dependencies and context variables, so let’s remove the feature flag
and fix forward for any issues.
Adds SAFETY comments for the two instances of unsafe blocks that are *not* just
FFI. It seems like overkill to annotate all the FFI calls so i'm skipping that,
let me know if anyone has strong preferences otherwise.
This is mostly to test that the new configuration skips the Rust CI step if
there are no rust changes - yay, that worked! But also worth mentioning in the
readme so lets land.
Adds name resolution support for class declarations and expressions. Mostly this
involves _not_ visiting some `Identifier` nodes that don't actually represent
variables. For example, method names don't introduce new variables (but if they
are computed, they may _refer_ to variables).
Adds an option to pass a list of known globals into the semantic analyzer so
that references to globals can be checked. As a follow-up we'll need to
distinguish between different types of semantic analysis errors, so that callers
which don't want to validate globals can ignore "unknown variable" reference
errors while still handling definite errors such as duplicate declarations.
JavaScript has ~sane~ fun rules around where variables can be redeclared or not,
and which kinds of variables this applies to. Actually the rules are pretty
straightforward:
* `var` can be redeclared any number of times.
* In strict mode, other declarations cannot be redeclared within the same scope.
This implies that a `var` declaration cannot conflict with these other forms,
which must take into account hoisting. So you can't have a `var a; let a` in the
same scope, but you also can't have a `let a` at a scope and then a `var a`
which will hoist to that same scope.
The [temporal dead
zone](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/let#temporal_dead_zone_tdz),
often abbreviated TDZ, is the period between the start of its declaring block
and the line that contains the let/const/class declaration. Not all instances of
TDZ can be detected statically, because whether or not a TDZ error will occur at
runtime is a property of which control flow path is taken and whether some other
code has initialized the value. However, a subset of cases can be detected, and
that's what we implement here.
When we encounter a variable reference we record the next declaration id at that
point in time. Then when resolving references after visiting the program, we can
check: did the reference end up referring to a let/const/class binding whose id
is equal or greater to that "next declaration"? If so, it means the reference
refers to a variable that is provably declared later and is a known TDZ
violation. The catch is that when resolving references, we reset the "next
declaration" limit value when we bubble up out of a function scope. That's
because references to let/const within a function may occur after the
declaration, and we can't statically validate them.
Previously we attempted to resolve each reference at the close of its defining
scope, and if it couldn't be resolved yet we bubbled the unresolved reference up
to the parent scope. That approach isn't ideal for two reasons:
* First, it's inefficient since we may have to make multiple attempts to resolve
the same reference.
* Second, it's incorrect. There can be cases where we think we can resolve a
reference to a value defined in an outer scope, but there is a hoisted
declaration from an intermediate scope that we haven't seen yet.
The safest and most optimal thing is to just queue all references and resolve
them at the end.
Rather than always create a Global scope and then immediately add a Module child
node, we now set the kind of the root scope to global (for scripts) or module
(for modules) based on the program node.
Teaches the semantic analyzer about import statements. We now treat imports as
declarations, so that subsequent references to the imported value can be
resolved.
Teaches the hermes->estree conversion to convert source ranges. This means our
diagnostics now point to the actual source of the error:
<img width="903" alt="Screenshot 2023-08-14 at 5 39 50 PM"
src="https://github.com/facebook/react-forget/assets/6425824/2960d114-0cc6-4531-9e7d-00ff3bfb1eb3">
Of course I still need to teach our semantic analysis about imports, but that's
a separate issue!
---
I added ~20 more tests to Sprout to get more of a feel for what the test
framework would need to support all fixtures. I'm relatively confident that the
approach outlined in [the original workplace
post](https://fburl.com/workplace/ftu8woch) works to migrate almost all existing
fixture tests to sprout (TLDR: use shared functions when possible, otherwise
write helper functions in-file).
Add `shared-runtime` file to reduce the amount of overhead needed to set up a
fixture test.
- All generalizable functions and values should be added here (e.g.
`shallowCopy`, `deepCopy`, `sum`).
- Editor integration is set up through a custom tsconfig, so importing from
`shared-runtime` should just work (with hover annotations, click-to-definition,
etc).
<img width="682" alt="Screenshot 2023-08-17 at 11 09 47 AM"
src="https://github.com/facebook/react-forget/assets/34200447/78c3dff9-ba10-4057-b3f6-2fa842d19b1d">
---
Tested new test fixtures added to sprout by adding them to `testfilter.txt` and
running with `--verbose --filter`.
Note that there are no unexpected exceptions, and all logs + returned values
match.
```
feifei0@feifei0-mbp babel-plugin-react-forget % yarn sprout:build && yarn sprout
--verbose --filter
yarn run v1.22.19
$ yarn workspace sprout run build
$ rimraf dist && tsc --build
✨ Done in 2.07s.
yarn run v1.22.19
$ node ../sprout/dist/main.js --verbose --filter
PASS console-readonly
ok {"a":1,"b":2} [
"{ a: 1, b: 2 }",
"{ a: 1, b: 2 }",
"{ a: 1, b: 2 }",
"{ a: 1, b: 2 }",
"{ a: 1, b: 2 }"
]
PASS constant-propagate-global-phis-constant
ok <div>global string 0</div>
PASS constant-propagate-global-phis
ok <div>global string 1</div>
PASS dce-loop
ok 10
PASS destructure-capture-global
ok {"a":"value 1","someGlobal":{}}
PASS destructuring-mixed-scope-and-local-variables-with-default
ok {"media":null,"allUrls":["url1","url2","url3"],"onClick":"[[ function
params=1 ]]"}
PASS holey-array-expr
ok [null,"global string 0",{"a":1,"b":2}]
PASS infer-global-object
ok {"primitiveVal1":2,"primitiveVal2":null,"primitiveVal3":null}
PASS infer-phi-primitive
ok 1
PASS infer-types-through-type-cast.flow
ok 4
PASS issue933-disjoint-set-infinite-loop
ok [2]
PASS jsx-tag-evaluation-order-non-global
ok <div>StaticText1<div>StaticText2</div></div>
PASS jsx-tag-evaluation-order
ok <div>StaticText1string value 1<div>StaticText2</div></div>
PASS method-call
ok 4
PASS mutable-lifetime-loops
ok {"a":{"value":6},"b":{"value":5},"c":{"value":4},"d":{"value":6}}
PASS mutable-lifetime-with-aliasing
ok {"b":[{}],"value":[{"c":{},"value":"[[ cyclic ref *0 ]]"},null]}
PASS update-expression-in-sequence
ok [4,2,3,4]
PASS update-expression-on-function-parameter
ok [4,1,4,2,4,3,1,4,4]
PASS update-expression
ok {"x":1,"y":1,"z":2}
PASS useMemo-multiple-if-else
ok 2
20 Tests, 20 Passed, 0 Failed
✨ Done in 4.11s.
```
---
This PR is an example of how to update fixture to add sprout annotations.
Running with `yarn sprout --verbose --filter` shows the fixture outputs.
```
PASS array-access-assignment
ok [[2],[[2],[],[3]]]
PASS array-expression-spread
ok [0,1,2,3,null,4,5,6,"z"]
PASS array-map-frozen-array
ok [[],[]]
PASS array-map-mutable-array-mutating-lambda
ok [[],[]]
PASS array-pattern-params
ok [{"a":"val1"},{"b":"val2"}]
PASS array-properties
ok {"a":[[1,2],2,"hello"],"x":3,"y":"[[ function params=1 ]]","z":"[[ function
params=1 ]]"}
PASS array-property-call
ok {"a":[1,2,"hello",42],"x":4,"y":1}
PASS assignment-expression-computed
ok [7]
PASS assignment-expression-nested-path
ok {"b":{"c":6}}
PASS capturing-func-mutate-2
ok {"a":2}
PASS capturing-function-alias-computed-load-2
ok "val2"
PASS capturing-function-alias-computed-load-4
ok "val2"
```
---
Rename `SproutOnlyFilterTodoRemove.ts` to `SproutTodoFilter.ts`. I've tried to
group the skipped fixtures by difficulty and add comments about what need to be
done, also happy to change the structure of this.
From this point, sprout will run on all new fixtures by default. If the fixture
forgets to export a `FIXTURE_ENTRYPOINT`, sprout will fail with this error.
<img width="853" alt="Screenshot 2023-08-15 at 5 59 32 PM"
src="https://github.com/facebook/react-forget/assets/34200447/0f80d650-1dc1-4df4-9710-e49acbb424b0">
See #1961 for an example of how to annotate existing skipped fixtures or new
ones. The `README.md` is also updated with a guide.
---
```
yarn sprout --verbose
```
Verbose mode prints out all outputs of tests. The test output is a status (`ok`
or `exception`), a returned or thrown value, and a set of console logs.
Currently as of #1960 , this is the output:
```sh
$ yarn workspace babel-plugin-react-forget run build && node
../sprout/dist/main.js --verbose
$ rimraf dist && tsc
PASS alias-nested-member-path
ok {"y":{"z":[]}}
PASS assignment-variations-complex-lvalue
ok {"y":{"z":4}}
PASS assignment-variations
ok 1
PASS chained-assignment-expressions
ok {"z":null}
PASS computed-call-evaluation-order
ok {"f":"[[ function params=0 ]]"} [
"A",
"B",
"arg",
"original"
]
PASS const-propagation-into-function-expression-primitive
ok 42 [
"42"
]
PASS constant-propagation-for
ok 0
PASS constant-propagation-while
ok 0
PASS constant-propagation
ok -6 [
"foo"
]
PASS controlled-input
ok <input value="0">
PASS do-while-continue
ok [1.5,1,0.5]
PASS do-while-simple
ok [6,4,2]
PASS expression-with-assignment
ok 5
PASS for-of-break
ok []
PASS for-of-conditional-break
ok []
PASS for-of-continue
ok [0.5,1,1.5]
PASS for-of-destructure
ok [0,2,4]
PASS for-of-simple
ok [0,2,4]
PASS function-declaration-reassign
ok {}
PASS function-declaration-redeclare
ok "[[ function params=0 ]]"
PASS lambda-reassign-primitive
ok 41
PASS lambda-reassign-shadowed-primitive
ok {}
PASS property-call-evaluation-order
ok {"f":"[[ function params=0 ]]"} [
"A",
"arg",
"original"
]
PASS reactive-scope-grouping
ok {"y":[{}]}
PASS sequentially-constant-progagatable-if-test-conditions
ok "ok"
PASS simple-function-1
ok "[[ function params=1 ]]"
PASS ssa-complex-multiple-if
ok
PASS ssa-complex-single-if
ok
PASS ssa-for
ok 11
PASS ssa-if-else
ok
PASS ssa-objectexpression-phi
ok {"x":1,"y":3}
PASS ssa-property-call
ok {"x":[[]]}
PASS ssa-property
ok {"x":[]}
PASS ssa-return
ok 2
PASS ssa-simple-phi
ok
PASS ssa-simple
ok
PASS ssa-single-if
ok
PASS ssa-switch
ok
PASS ssa-throw
exception undefined
PASS ssa-while
ok 10
PASS type-field-load
ok 1
PASS type-test-field-store
ok {}
PASS type-test-primitive
ok 2
PASS update-expression-constant-propagation
ok {"a":0,"b":0,"c":2,"d":2,"e":0}
44 Tests, 44 Passed, 0 Failed
✨ Done in 9.27s.
```
---
Not sure why, but snap kept silently crashing when I used fs/promises to write
to many file handles. I tried a `.catch(...)`, but couldn't figure it out. This
diff changes snap to use sync fs apis to avoid crashes, but I'd love to get
feedback if someone knows how to debug this.
This PR moves the phi evaluation to a separate function.
Most importantly, it inverts the default case to _not_ constant propagate unless
we have explicit validation of the phi operands.
---
## Sprout 🌱 Overview
**(Overview copied from
[sprout/README.md](0468ddf8bb/forget/packages/sprout/README.md))**
React Forget test framework that executes compiler fixtures.
Currently, Sprout runs each fixture with a known set of inputs and annotations.
We hope to add fuzzing capabilities to Sprout, synthesizing sets of program
inputs based on type and/or effect annotations.
Sprout is currently WIP and only executes files listed in
`src/SproutOnlyFilterTodoRemove.ts`.
### Milestones:
- [✅] Render fixtures with React runtime / `testing-library/react`.
- [ ] Make Sprout CLI -runnable and report results in process exit code.
After this point:
- Sprout can be enabled by default and added to the Github Actions pipeline.
- `SproutOnlyFilterTodoRemove` can be renamed to `SproutSkipFilter`.
- All new tests should provide a `FIXTURE_ENTRYPOINT`.
- [ ] Annotate `FIXTURE_ENTRYPOINT` (fn entrypoint and params) for rest of
fixtures.
- [ ] Edit rest of fixtures to use shared functions or define their own helpers.
- [ ] *(optional)* Store Sprout output as snapshot files. i.e. each fixture
could have a `fixture.js`, `fixture.snap.md`, and `fixture.sprout.md`.
### Constraints
Each fixture test executed by Sprout needs to export a `FIXTURE_ENTRYPOINT`, a
single function and parameter set with the following type signature.
```js
type FIXTURE_ENTRYPOINT<T> = {
// function to be invoked
fn: ((...params: Array<T>) => any),
// params to pass to fn
params: Array<T>,
// true if fn should be rendered as a React Component
// i.e. returns jsx or primitives
isComponent?: boolean,
}
```
Example:
```js
// test.js
function MyComponent(props) {
return <div>{props.a + props.b}</div>;
}
export const FIXTURE_ENTRYPOINT = {
fn: MyComponent,
params: [{a: "hello ", b: "world"}],
isComponent: true,
};
```
---
## Implementation Details
- jest-worker test orchestrator (similar to Snap 🫰).
I chose to write a test runner instead of directly using Jest for flexibility
and speed.
- Sprout 🌱 currently runs much more code per fixture (2 babel transform
pipelines + 2 `exec(...)` than snap, so all scaling concerns apply.
- Sprout may need more customization in the future (e.g. fuzzing component
inputs, caching artifacts)
- We probably want to add snapshot files for Sprout, which is much easier with a
custom runner.
This is also one of the main reasons we wrote Snap. Jest consolidates all
external snapshots (i.e. non-inline snapshots) from a test into [a single
file](d0a006ffa9/forget/src/__tests__/__snapshots__/compiler-test.ts.snap).
This was painful mainly for rebasing changes, but also for small papercuts (e.g.
needing to run `yarn test -u` twice when deleting a fixture)
- Currently does not save output to snapshot file, but we can easily add this
later (i.e. each test would have a `.snap.md` and `.sprout.md` file)
- Supports filter mode (same testfilter.txt file as snap)
- Currently does not support watch mode. I expect that sprout's primary use will
be catching bugs in the PR / Github Actions phase. Can be changed if we need to
iterate on sprout output while developing.
- All tests are run with `react-test-renderer` to access the React Runtime,
which is needed for calls to `useMemoCache` (and other potential hooks).
- react-test-renderer required a mocked DOM, so I ~~used js-dom~~ did a terrible
js-dom hack to add document, window, and other browser globals to the
jest-worker globals, then exec the test code in the jest worker global.
I can clean this up later by bundling library code (e.g. react,
react-test-renderer) to not use `require(...)`, then calling `exec` on all "test
client code" in the js-dom mock global (instead of the real jest worker one)
- Tests marked `isComponent` need to return valid jsx or primitives. Values
returned by all other tests are converted to a string primitive via
JSON.stringify.
```sh
# install new dependencies to node_modules
$ yarn
$ cd forget/packages/babel-plugin-react-forget
$ yarn sprout:build && yarn sprout
PASS alias-nested-member-path
PASS assignment-variations-complex-lvalue
PASS assignment-variations
PASS chained-assignment-expressions
PASS computed-call-evaluation-order
PASS const-propagation-into-function-expression-primitive
PASS constant-propagation-for
PASS constant-propagation-while
PASS constant-propagation
...
✨ Done in 3.91s.
```
Nice find from @mofeiz, we weren't adding declarations for function params in
LeaveSSA. This caused us to hit an invariant for update expressions that updated
params which assumed that all named variables had a declaration. This is why
it's helpful to add invariants, it helps you find places where you violate them
:-)
Adds basic support for hoisting semantics: * Resolution of variable references
is _always_ deferred in case the correct binding hasn't been seet yet due to
hoisting. * Var and function declarations bubble to the appropriate scope
There are lots of subtleties that aren't implemented yet but these rules cover a
lot.
This is a precursor to adding support for hoisting in semantic analysis.
Previously when we encountered an unknown reference we immediately reported an
error. But hoisted variables may be referenced before they're defined, so we
don't know for sure when we see an unknown variable if its actually unbound or
not.
This PR adds the first part of hoistingn support: rather than immediately report
an error when encountering an unbound variable we store it in a list of
unresolved references on the current scope. As we close each scope we recheck
and see if the variable can now be resolved. If yes we record that, otherwise we
bubble up the unresolved reference to the parent scope (and try again there).
The next PR(s) will handle hoisting of `var` and other syntax to the apropriate
nearest scope boundary (function/module).
When updating the data model for operands from instruction indices to
identifiers, I forgot to rewrite terminal operands.Doing so required a refactor
to use the new BlockRewriter helper.
Adds a `Destructure` instruction closely following the design of the TS-based
compiler. The main change is fairly small: the TS compiler doesn't support rest
spreads that are not identifiers, such as the `...y[]` in `const [x, ...[y]] =
z`. I added support for this in the Rust compiler.
`compileProgram` was getting complex, so this extracts some of the logic into
smaller functions. Additionally, the `try` block now only wraps the `compileFn`
generator from Pipeline, which means not accidentally catching other non-Forget
errors
Initial data types for `forget_reactive_ir`, which is the Rust analogue of
`ReactiveFunction` in the TS compiler. I'm renaming here for clarity, though
naming suggestions are very welcome!
Aligns the data model for `Instruction` closer to JS: * Adds an `lvalue:
IdentifierOperand` property (Identifier + Effect) * Changes rvalues from being a
reference to the definining instruction by instruction offset (InstrIx) to be
a variable reference (IdentifierOperand)
There are pros and cons to the previous offset-based approach. It's definitely
convenient to be able to jump directly to the instruction that defined an
operand value. However: * Many passes need to track things like basic
reassignments, which mean they need to build up mappings of IdentifierId to
some data. When all operands are IdentifierOperands, this can be a single
mapping. * It aligns closer to JS, which makes porting a bit easier.
But most of all: porting the `ReactiveFunction` data type — which is in tree
form - is non-trivial with the offset-based approach. As we map HIR to the tree
form, we'd have to remap every operand offset. Using identifiers for operands
simplifies this.
We can always revisit this design choice later.
After the previous PR we no longer depend on SWC for the critical path of
development/testing. Notably this unblocks adding support for the rest of the
language: the new `forget_hermes_parser` crate automatically converts Hermes
Parser's AST into our `forget_estree` format via codegen. Achieving full
language support with SWC would have required manually defining the remaining
conversions for the rest of the language.
Long-term we'll need to revisit how to integrate into SWC, and more generally
into setups build atop SWC such as Next.js's Turbopack-based build
configuration. But i'll delete for now since we don't depend on it for
iteration, it's slow to build, and requires us to opt-in to nightly Rust.
Replaces the use of SWC parser in the Forget fixture tests with Hermes Parser.
This includes aligning on a single type to represent JS Values and numbers
(combining semi-duplicated code from forget_hir and forget_estree) and adding
support for lowering babel-style
NumericLiteral/BooleanLiteral/StringLiteral/NullLiteral node types (since Hermes
Parser produces a mix of estree/babel node types)
Updates HIR builder to rely on the new semantic analysis instead of assuming
that the ast nodes will already have binding info attached. That was a stopgap
until we had our own name resolution :-)
Once this lands we can remove SWC and switch everything to forget_hermes_parser,
and also remove the non-spec Identifier.binding field (which stored the
temporary name resolution data).
Adds new instructions to accurately model UpdateExpression semantics, since
`x++` is un-intuitively not the same as `x = x + 1`. There are a few different
ways to model the combination of prefix/postfix and increment/decrement:
* One instruction for all combinations of prefix/postfix and
increment/decrement, eg 'UpdateExpression'
* Instructions for Increment/Decrement, each with a property to distinguish
prefix/postfix
* Instructions for Prefix/Postfix, each with aproperty to distinguish
increment/decrement.
I chose the latter, `PrefixUpdate` and `PostfixUpdate`, because it keeps the
number of new instructions minimal while keeping separate instructions for the
most important distinction: whether the result of the instruction is the value
before applying the operation or after. I'm open to suggestions about this
though.
A few quick notes:
* Constant propagation is supported but only for numbers (we don't support
bigint yet anyway)
* LeaveSSA needs to know about these instructions since their presence requires
making the original variable declaration Let, not Const.
* EnterSSA mapped lvalues before rvalues, which is out of order but didn't
previously matter. I just had to flip the order and everything worked.
The for loop over eachInstructionLValue already rewrites instr.lvalue: ```
for (const place of eachInstructionLValue(instr)) {
rewritePlace(place, rewrites); } ```
Adds semantic analysis support for normal `for` statements and for JSX. The main
catch with JSX is that there are a bunch of identifiers that we have to ignore
since they aren't variable references: jsx attribute names, namespace names, jsx
member expression properties, and closing elements.
Updates `estree` codegen to emit a Visitor trait (temporarily named `Visitor2`
since there is a hand-rolled one that some code is using). We use knowledge of
the grammar to only visit fields whose type is a Node or Enum, or an "object"
type that opts into being visitable. The latter is used for Function and Class.
This will make it much easier to write the semantic analyzer.
This is two things:
* A toy semantic analysis that handles a tiny subset of JS, including labeled
statements, labeled break/continue, and variable
declaration/reference/reassignment. This only exists as a way to prove out the
API for the more important bit:
* More importantly, this defines a data model for the semantic analysis results
and an API for building up the semantic analysis.
Subsequent diffs will replace the first bit (toy analysis impl), while keeping
the second part.
---
Changes in `@enableOptimizeFunctionExpressions` caused a bug in the last Forget
sync to VR Store. The repro can be summarized to something like this:
```js
function foo() {
const x = true; // some constant or global
// Add some branching for type inference
// This can be a Logical expression as well (e.g. `4 || 5`)
if (...) { }
// In this HIR block, SSA inserts a `x$2 = phi(x$1, x$1)`.
// EliminateRedundantPhiNodes needs to rewrite all references of `x$2` to `x$1`
const accessXInLambda = () => x;
return accessXInLambda;
}
---
Revives e2e test infra from #587.
- All React component-like functions are compiled.
- `yarn jest` runs each e2e test twice (forget and no forget)
Github Actions is already running `yarn test`, which includes all jest tests
```
Run yarn test
yarn run v1.22.19
$ yarn workspaces run test
> babel-plugin-react-forget
$ yarn jest && yarn snap:build && yarn snap
$ tsc && jest
PASS main src/__tests__/Result-test.ts
PASS main src/__tests__/DisjointSet-test.ts
PASS e2e with forget src/__tests__/e2e/hello.e2e.js
PASS e2e no forget src/__tests__/e2e/hello.e2e.js
Test Suites:
[4](https://github.com/facebook/react-forget/actions/runs/5732016200/job/15534129231?pr=1881#step:8:5)
passed, 4 total
Tests: 23 passed, 23 total
Snapshots: 11 passed, 11 total
Time:
6.1[5](https://github.com/facebook/react-forget/actions/runs/5732016200/job/15534129231?pr=1881#step:8:6)3
s
```
Using an arena allocator can be faster, but it comes with several challenges:
* It requires tediously tagging nearly every value with a lifetime. Any function
that has to deal with arena-allocated data (which is every meaningful function)
ends up with a lifetime parameter. Bleh. We also have to thread the allocator
itself wherever we need to allocate, though this is less of a problem since
_most_ functions already need the Environment and we can store the allocator
there.
* There is not yet widespread support in the Rust ecosystem for using custom
allocators with custom data types. This means that things like HashMap/Set and
IndexMap/Set can't be arena allocated. This means that either we have to add
support to these data types (by upstreaming or forking) or just accept that
we're only partially using the arena allocator.
* Finally, `bumpalo`'s `Box` cannot be moved out of, which turns out to be an
annoying limitation that i've already had to work around several times.
In the end i'm not sure arena allocators are worth it at this stage of the
project. Relay Compiler has been successful without one, and other Rust-based
internal compilers get by without them too.
The current heuristic to check if setState is called in render is based on
whether the lambda containing the call to setState has a mutable range that got
extended. This doesn't seem to work in all cases so this validation needs a bit
more work before we can turn it on by default
This includes a fix where the HermesToBabelAdapter was incorrectly outputting a
`ClassMethod` instead of `ClassPrivateMethod` in certain scenarios. This was
causing issues with our eslint plugin as it would crash on any file containing
private methods because a malformed AST was formed.
There was a bug in our internal Hermes to Babel adapter which caused some
malformed AST to be constructed, which babel would then validate as being
incorrect. That bug was fixed, but we still have to add this plugin so that
private class methods can be parsed by babel.
Tested internally
Adds GitHub actions to build and test the Rust version of Forget on every commit
to main. I didn't enable this on PRs for now just to avoid slowing down other
folks, this seemed like a reasonable compromise while we're still in the
experimentation phase. I test locally, and CI will catch if I or anyone else
slips up.
Rejects unknown fields during deserialization of helper structs (which don't
have a `type` field). This would be super useful for AST `Node` types, too, but
that requires implementing a custom deserializer so i'm punting on that for now.
Adds more AST types to handle the full ES2021 spec. At least, in theory I
defined the schema correctly, and we'll have to just fix any bugs as we
encounter them.
Updates to use our own codegen'd `Serialize` implementation for AST node types.
Notably, this allows us to ensure that we always emit a `type` field, even when
the type is obvious from context (serde doesn't do this) and avoid
double-emitting `type` fields (which serde can do if you try to force a tag to
be emitted). We still use the Deserialize impl because it works fine for our
purposes.
Adds almost all the remaining AST types from ES2015 to `estree`, and updates the
swc->estree->hir conversions accordingly. In a few places i punted with a
`Diagnostic::todo()`.
Ports InlineUseMemo to Rust, this is the only missing transform pass on the
pipeline up through constant propagation. I wanted to finish this so that we
could do benchmarking of the early phase of compilation. UseMemo does a bunch of
rewriting so it seemed worth comparing performance.
Included:
* Add swc -> estree and estree -> HIR conversions for arrow functions and call
expressions
* Add HIR definition for labeled terminals and call expressions
* Utility for inlining one function into another — note that this requires
remapping InstrIx operands since instruction indices will change. An alternative
would be to store all the instructions for both outer/inner functions in the
same array, in which case we wouldn't need to remap.
* Helpers for mutably iterating the operands of an instruction or terminal.
* The actual inline_use_memo() function. The overall logic is similar, i just
had to slightly shuffle the order of operations to satisfy the borrow checker.
Console warnings when running tests: ``` ts-jest[versions] (WARN) Version 5.1.3
of typescript installed has not been tested with ts-jest. ```
Our typescript version is ahead of what's supported in ts-jest. This PR updates
ts-jest to work with the latest typescript compiler.
This PR adds a Hole kind that can be present in both ArrayExpression and
ArrayPatterns.
This Hole type is not interesting for our inference passes and is skipped over
for all of the pipeline.
Forget doesn't understand the React namespace object and generates incorrect
code when compiling code that loads props from this namespace object.
This PR makes Forget bailout when we see a property load from React namespace
object.
Upgrades every dependency on `hermes-parser` and `prettier` to the versions that
we're using in the rest of our other codebases. Notably, these package versions
are necessary for our other codebases to make use of mapped types in Flow.
8d021e5d797c5ef3b4e18a0be735c04005f2ae7a configured `./forget` as a workspace
root, so these `yarn.lock` files in `app/*` and `packages/*` are no longer
relevant. This PR deletes them to reduce noise and confusion.
Ports `MergeConsecutiveBlocks` to Rust. This was a tricky one: as we iterate
through the blocks _if_ the block ends up being merged with its predecessor we
need to consume it and modify its predecessor block (ie, mutating two things
from the same data structure - shared mutation!). But if we _don't_ need to
merge, then we need to not drop the current block. Ie, we sort of need to
conditionally take ownership of the current block during iteration and put it
back.
I added a `BlockRewriter` helper type for this which has a helper to iterate
safely. It calls the iterator lambda, moving blocks one at a time into the
lambda. The lambda returns either `Keep(block)` to give the block back and keep
it or `Remove` to tell the rewriter to drop the block. Thanks to making the
Blocks data structure hold `Option<Box<BasicBlock>>` items, "moving" the block
actually just means nulling out the option and conceptually giving ownership of
the pointer to the callback - the data itself never moves.
This involved creating a custom `Blocks` wrapper type, which i had been putting
off doing. This cleans up a bunch of other logic around traversing blocks.
This PR adapts the `Diagnostic` type and helpers from Relay Compiler to Forget.
The main changes are:
* Removing some fields it doesn't seem we'll use for a while, if ever (like
machine-readable arbitrary key/value data)
* Switching from Relay Compiler's `Location` type to our SourceRange type
* Using the severity enum previously established in the `forget_build_hir`
crate, with Todo/Unsupported/InvalidSyntax/InvalidReact/Invariant variants
* Adding support for translating our `Diagnostic` into a `miette::Diagnostic` so
we can use miette's pretty printing
With the new Diagnostic type in place i updated the existing build_hir code to
use it and confirmed that the errors are now even nicer (when we attach extra
data to annotate labels):
<img width="860" alt="Screenshot 2023-07-14 at 3 10 00 PM"
src="https://github.com/facebook/react-forget/assets/6425824/9d29425a-938b-4872-b999-aa174a3c329a">
This addresses (or brings us closer to addressing) many of your comments on the
last diagnostics PR, @poteto!
Per the title, this updates the main readme file with a guide to contributing to
the Rust compiler, and ensures that we have a brief description of every crate
in local readme.md files. The `forget_hir` one is the most extensive and
describes the high-level design of the HIR.
I've primarily used what Cargo calls virtual workspaces, where the top-level
Cargo.toml just lists a bunch of packages and they each have their own
dependencies. This is fine, but i've noticed that more repos are using real
workspaces and they offer a bunch of benefits. You define the dependencies in
the top-level Cargo.toml and then can easily refer to them from multiple crates,
ensuring all the versions match up. It makes it easier to refer to other crates
in the workspace, too, because you define the path once at the root, then every
other crate can just say `forget_foo = { workspace = true }`.
I also renamed all the crates to be prefixed with `forget_`, in some cases
removing the redundant `hir` name, So `hir-optimization` became
`forget_optimization`, `hir-ssa` became `forget_ssa`. Also note the switch from
hyphenated names to underscores everywhere, since at the end of the day you have
to write the name with underscores in source code.
I also deleted the demo crate that i started with since we don't need it
anymore.
And finally, i added an explicit publish = false to all the crates just to
prevent mistakes.
This is a quick "good enough" first pass at computing function expression
context variables. It definitely needs to be overhauled, but it's enough to make
a lot of common cases work correctly.
First, this PR adds a hand-rolled Visitor trait for `estree`. Long-term that
should probably be code-generated, but there are some subtleties to it such as
the `visit_lvalue(callback)` helper which has to be wrapped around various calls
(or we need some other way to distinguish identifiers within lvalues from
identifiers within rvaluess). So for now it makes sense to hand-roll it until we
are more confident in exactly how it should work.
Given that visitor, i was able to port part of the existing
`gatherCapturedDeps()` to Rust with some modifications. Note that we assume
_something_ has run name resolution on the estree to match up identifiers to the
declaration they refer to. But we don't store information about parent scopes so
we can't walk up to check where things were defined. Instead we do the
following:
* Build up a list of all referenced identifiers
* Also build up a set of bindings defined in the function itself
* After visiting, filter our the first list to only include identifiers not
defined by the function itself, and to eliminate duplicates.
This covers the majority of cases: the most obvious gap is nested function
expressions though actually that should just work.
Updates eliminate_redundant_phis and constant_propagation to recurse into
function expressions. I also realized there was a bug in EliminateRedundantPhis
in which we wouldn't traverse into function expressions encountered after
finding a back edge, so i fixed that logic in both versions.
Updates `enter_ssa()` to recurse into function expressions. In the TS compiler
we use a single Builder instance and copy (references to) the function
expression's blocks into the builder. That type of sharing just does not play
nicely with Rust. But... we don't need to do that! We already know the context
variables of the function expression, so we can lookup each of them to find
their re-mapped identifier, and set that as the starting state for the entry
block of the function expression. That lets us use normal recursion and
otherwise not share any information between the outer and inner builders.
Of course to make this work we actually have to populate Function.context, but
the algorithm _should_ work.
Start of function expression support:
* Basic structure for representing function expressions in the HIR
* Printer support
* swc -> estree -> hir conversion for function expression _bodies_. Dependencies
and context are not handled yet.
Makes the same improvement to constant propagation in both the JS and Rust
versions. The core algorithm only populates phi variables if all operands have a
known value (no back edges) and all those values are the same: this allows us to
propagate constants in most cases and simply punts on handling propagating
values that are affected by loops. However, since we collapse if statements into
gotos when the test condition is a constant, there can be cases where a phi that
originally existed will be pruned out:
```
// bb0
let x1;
if (true) {
// bb1
x2 = 1;
} else {
// bb2
x3 = 2; // this block becomes unreachable
}
// bb3
x4 = phi(bb1: x2, bb2: x3); // this phi will get pruned s.t. x4 = x2 = 1
return x4;
```
However, the algorithm doesn't prune phis until _after_ applying constants, so
currently we would see this phi node w different inputs and not propagate a
constant for the final usage of x, even though it will clearly be `1`.
The change is to make constant propagation use fixpoint iteration, iterating so
long as terminals changed on the previous iteration. If no terminals change the
algorithm completes in a single pass, but if terminals do change then we update
phis and continue. As you can see from the new test case this allows us to find
arbitrary length sequences of values and terminals that can be pruned.
Ports constant propagation to Rust. The algorithm is broadly similar to the TS
version, and most of the differences come from the slightly different HIR data
model (operands are instruction indices not identifier ids). What this means is
that the Constants map that we build up is really only used for variables that
existed in the original program, and only comes into play with instructions like
LoadLocal and StoreLocal. Other instructions such as Binary just look up their
operands directly, ie they load the referenced instruction to check if both
left/right are primitives.
Note that with SSA form and the index-based operands we could actually get rid
of StoreLocal/LoadLocal completely, which would further simplify constant
propagation. However:
* we'd need to add a Phi instruction kind, not a big deal but it diverges even
more
* more importantly, it makes it super hard to implement LeaveSSA
That second point is a deal-breaker so unless someone has a great idea for how
to exit SSA form without having Load/Stores, let's keep them.
This is a nearly 1:1 port of EliminateRedundantPhis to Rust, the algorithm is
identical and all differences are superficial. There are few things missing (an
invariant instead of a panic in one place, recursing into function expressions)
but the Rust version is still going to end up shorter despite keeping all the
comments.
This is a first pass at porting EnterSSA to Rust. First pass in the sense that
it's hard to fully test it, and also in the sense that we'll likely figure out
even better ways to work w the HIR as we iterate. Oh and i didn't do recusing
into function expressions yet, since we can't even represent function
expressions yet, and that may require modifying the design a bit (though i have
an idea that i think will work, which is for the Builder to have an optional
parent. When we encounter a block with no predecessors, we check the parent. I
_think_ this will make all the borrowing "just work").
A few notes:
Rust's compilation model parallelizes and incrementally computes at the crate
granularity, so builds are faster if we split up our code into more but smaller
crates. Setting up a clean dependency graph can dramatically improve build
performance too. For example, to run the `fixtures` tests we can build `estree`
and `hir` in parallel, then once those build _all_ of our other crates can be
built in parallel until we get to `fixtures` which depends on the everything
else. The various passes don't have any build dependencies on each other so they
can build in parallel. Hence the new code for SSA stuff is in a separate
`hir-ssa` crate. We should similarly group other passes (approximately one crate
per folder in the babel-plugin-react-forget/src/ directory, eg SSA, Inference,
Optimization, etc).
Second, shared mutable ownership can be modeled in Rust but requires wrappers
such as `Rc<RefCell<>>`. It's generally more efficient and more idiomatic to
rethink the data model and algorithm. For EnterSSA, the Builder object holds a
reference into the HIR that it only ever reads, and the pass (which drives the
builder) also holds a reference into the HIR, which it mutates. The previous PR
split up Blocks and Instructions, and the value of that is more apparent in
`enter_ssa()`. The Rust equivalent of the builder holds a _shared_ (immutable)
reference to just the HIR's blocks, while the pass (driving the builder) holds a
_unique_ (mutable) reference to just the HIR's instructions. This lets us keep
the overall feel of the algorithm while keeping Rust happy.
Also note that the other change — to making operands be InstrIx indices into the
instructions array — means that the SSA logic is simpler. Most instructions
don't have to be visited at all, since they don't deal with loads/stores.
Terminals also don't need to be visited, since they reference instructions, not
identifiers. The Phi concept seems to just work too.
I also updated the printer to print predecessors and phis.
This change is motivated by starting to explore porting EnterSSA to Rust. It's a
good medium complexity pass and quickly demonstrates why a direct port of our
existing data model and algorithms won't work so well. For examples just these
first lines at the top of the transform create multiple references to the
function body/blocks:
58da89888e/forget/packages/babel-plugin-react-forget/src/SSA/EnterSSA.ts (L230-L231)
But more generally it's always felt wrong that instructions have an LValue that
isn't really used. So here i'm exploring making operands a newtype index into a
single instructions array (shared for the entire function), and using different
types for identifier references in SetLocal and LoadLocal. Incidentally this
also declutters the printed HIR quite a bit.
I'm not going to land this until i actually finish enter_ssa() and some other
passes. We definitely need to balance fidelity to the existing code (to
facilitate porting) with using idiomatic Rust (to facilitate porting in the
sense of not fighting the borrow checker).
This mode improves compilation and makes optimizations easier, let's make it the
default. I previously confirmed that enabling this mode didn't affect output
when synced internally, and I'll do that again before landing the PR.
I added this as a quick workaround, since we didn't support unused
logical/conditional expression statements. Now that we handle them we don't need
InstructionValue::ExpressionStatement anymore. I found this when porting our
lowering to Rust.
The previous PRs to make `estree` use codegen broke the swc->estree and
estree->hir conversions. This PR updates those conversions so everything builds
now.
The overall goal of this workstream is to have a Rust representation of ESTree
that we can use as the input and output of the compiler. In Rust environments we
can convert between the native AST of SWC or OXC and ESTree, and when invoked
from JavaScript we can serialize to/from ESTree-compliant JSON. Given that our
first target is to plug into a JS-based compilation toolchain, we need to have a
working serialization to/from ESTree JSON. The point of the codegen-based
`estree` crate is to allow us to model estree as ergonomic, idiomatic Rust (to
make consuming it in code easier) while also allowing us to serialize to/from
spec-compliant ESTree. This PR flushes out one remaining piece.
Updates our estree codegen to generate a custom `Deserialize` implementation
instead of using the derived one from serde. ESTree has some enums whose
variants are themselves enums, for example we have
```rust
enum ModuleItem {
ImportExportDeclaration(ImportExportDeclaration),
Statement(Statement),
}
enum ImportExportDeclaration { ... }
enum Statement { ... }
```
This sort of works with serde's derive implementation: you have to use the "tag"
representation for the inner enums, and an "untagged" representation for the
outer one (ModuleItem). The problem is that with an untagged representation,
serde doesn't know what type of data it's expecting. All it can do is go one by
one and try to parse the data as the first variant (eg ImportExportDeclaration)
then the next one (Statement) and fail when it gets to the end of the list. If
the data isn't valid for any reason, deserialization will fail with a "not a
valid ModuleItem" error. That's true but not helpful, especially if you're
developing estree, are confident that the input json is valid, and need to
figure out where you messed up the definition. It's also not helpful as an
end-user if you're not sure your input json is valid.
So this PR updates our codegen to emit a custom derive implementation that is
identical for both regular enums (like Statement) and recursive ones (like
ModuleItem). We first extract the tag to know what type the value is, then
deserialize exactly as that type. So in the above case, rather than have to
first try parsing every ModuleItem as an ImportExportDeclaration and then fall
through to statement, we just decode the tag (`type` in our case, for example
say it's an "ForStatement"), then deserialize directly as that type (eg, as
ForStatement), then wrap it in the enum variant (ModuleItem::Statement(...)).
For recursive enums like ModuleItem we add an extra wrapper as necessary.
The end result is that we get much more precise errors and deserialization is
more efficient: we always decode just the tag, then as exactly that type.
Note that our serialization is also not perfect right now, because we don't
always emit the `type` key. Serde only emits it when a value appears in an enum.
We can similarly generate custom serializers for all our types to always emit
the tag. That will be straightforward when it's necessary. The current PR was
more of a blocker, because it was really hard to figure out mistakes in the
estree definition given the ambiguous errors. Thanks to this PR we now get
precise errors along the lines of "unknown type `JSXElement`" which are easy to
resolve.
Uses the `syn` crate, which can parse various Rust syntax forms, to parse the
`type` field from json schema description. This allows us to describe complex
types like `"type": "Vec<Option<ArrayElement>>"` directly, rather than requiring
flags like nullable, plural, and nullable_item. The main flag that i'm keeping
is "optional", which is used to indicate when the field itself (not the value)
is optional.
Adds some parts of the ES2015 spec, such as imports and ForOfStatement. This is
enough to get a few more fixtures compiling. The last one uses JSX which I
haven't defined yet.
This is meant to replace the initial `estree` crate with a version that is
generated from a JSON description of ESTree. The idea is to make it easy to
experiment with slightly different representations to balance ergonomic usage of
the data at runtime with serialization compatible with ESTree spec. The JSON
schema looks like this (somewhat abbreviated):
```
{
// Objects are struct types that don't have a `type` and can't appear as an enum
variant
objects: {
Position: {
line: {type: "NonZeroU32"},
column: {type: "u32"}
},
...
},
// Nodes are struct types with a `type` and which can appear as enum variants
(statements, expressions, patterns, etc)
nodes: {
ArrayExpression: {
elements: {
type: "Expression",
plural: true,
nullable_item: true,
}
}
...
},
// Categories of nodes with multiple variants, represented as enums.
// Can be recursive, eg ForInit can be VariableDeclaration or Expression,
// where Expression is also an enum
enums: {
Expression: [
"ArrayExpression",
...
]
},
// Simple enums which have a corresponding string value. Used primarily for
operators (binary/unary/logical/etc)
// but also for things like variable declaration kind (var/const/let)
operators: {
BinaryOperator: {
Plus: "+",
Instanceof: "instanceof",
}
}
}
```
The core estree files are now generated using Cargo's build script mechanism.
Right now i only defined the types and fields from ES5, so i'll have to flush
out the rest of the modern JS spec and extensions like JSX, TypeScript, and
Flow. But already the for-statement example works, showing that this approach
can handle complex cases such as unions of types or other unions (ForStatement
initializer is tricky bc it can be a VariableDeclaration or an Expression - that
works now!).
This is still WIP a bit - now that the ESTree definition is more precise i can
go back and clean up some other code (have to, because the swc -> estree
conversion needs some tweaks now).
Until now i've freely used `panic!`, `unwrap()`, and friends for "error
handling". This PR switches to consistently returning `Result` within the HIR
builder, using a structured error representation that exploits helpers from
`thiserror` and `miette` crates. Miette has a super graphical formatter for
diagnostics as you can see in the screenshot (also see the
[repo](https://docs.rs/miette/5.9.0/miette/index.html)).
This is just a first pass and we'll need to flush out the error handing story
more. Two obvious directions to go next:
* Make HIR construction error-tolerant, so that it can find as many errors as
possible at once rather than failing on the first error. We did this in Relay
Compiler as well, and we can likely borrow some of its helpers.
* Decouple from `miette`. It's very nice but less flexible than I'd like. We can
define our own more generic diagnostic type that contains structured data, then
have a generic conversion mechanism into a miette type so we can use their
display logic.
<img width="789" alt="Screenshot 2023-07-06 at 3 55 38 PM"
src="https://github.com/facebook/react-forget/assets/6425824/e1f1ed4b-5188-4af5-9af4-8f6c5c345023">
Implements support for `ForStatement` from swc -> estree -> hir, flushing out
more of the HIR representation and porting pieces from HIRBuilder as necessary.
Fundamentally this PR is about lowering identifiers during construction of HIR.
For now i'm punting on context variables and assuming all variables are either
global, module-scoped, or locals. The implementation involves a few pieces:
* `estree::Identifer` gets extended with optional binding information. The idea
is that _some_ name resolution mechanism will populate this. Eventually our own,
but we can also borrow data from another source...
* `estree-swc` now configures SWC's (possibly broken?) name resolution mechanism
and sets the above binding data when translating identifiers from swc into our
estree format.
* hir `Builder` tracks identifiers based on `(name, BindingId)` pairs, and
assigns a unique `hir::Identifier` instance for each pair. `Identifiers` are
clone-able (shared).
* Tangential: i updated the printer to handle more instruction variants,
including the now-ported LoadGlobal instr.
I don't love this but it's a start. Long-term we definitely should have our own
name resolution mechanism which we run on the estree prior to lowering to HIR.
Implements a pretty-printer for the HIR and switches the fixture tests to use
this instead of the debug format. It's much more readable now!
Note that not all types are properly printed, I only implemented the
instructions and terminals used in the example. For others we fall back to the
Debug impl so we at least print something.
Adds a new `fixtures` crate intended for running end-to-end tests of the
compiler. As we expand the compiler this will eventually match our JS fixture
setup, where we have .js files as input and produce memoized JS output.
For now, this does the following:
* Parses with SWC (omg this was painful to setup)
* Runs SWC's name resolution, which annotates the SWC ast in-place
* Convert the SWC ast into our `estree` representation
* Convert `estree` into `hir` for each top-level function declaration in the
input program
* Print the Rust `Debug` view of the resulting HIR
As a next step i'll add a pretty-printer for the HIR to roughly match what we
have in JS.
Multiple Place instances can share a reference to a given Identifier in our JS
implementation. For simplicity of the initial port I’m using Rc (for sharing)
and RefCell (for runtime-checked mutability). This is the standard pattern for
shared mutable references in Rust when you don’t need multi-threaded support. We
don’t need HIR to be accessible by multiple threads so this is fine, if we do
multiple threads it will be to parallelize compilation of separate functions.
There are other idioms w less runtime overhead, such as Place holding an index
into a separate vec of identifiers, but that would make the port much less
straightforward.
Starts to port BuildHIR, in the Rust case this means the ESTree -> HIR
conversion. This necessitated flushing out the Builder struct a bit more. Mostly
the logic translates over very directly, and if anything it's cleaner because of
the lack of noise dealing with TypeScript unsoundness for Babel typedefs and
switch statements.
This is a start to porting HIRBuilder, with a largely complete implementation of
`build()`. Notably this includes all the passes which build() calls, and the
helper functions those passes call in turn:
```rust
reverse_postorder_blocks(&mut hir);
remove_unreachable_for_updates(&mut hir);
remove_unreachable_fallthroughs(&mut hir);
remove_unreachable_do_while_statements(&mut hir);
mark_instruction_ids(&mut hir)?;
mark_predecessors(&mut hir);
```
This was pretty straightforward. I ran into one borrow checker issue with (iirc)
`remove_unreachable_for_updates` where i needed to simultaneously hold a mutable
reference into the HIR (to write the ForTerminal) and also get an immutable
reference to check if the update block is reachable. The challenge is this:
```rust
for block in hir.blocks.values_mut() {
^^^^^^^^^^^^^^^^^^^^^^^ hir borrowed mutably here
if let TerminalValue::ForTerminal(terminal) = &mut block.terminal.value {
if let Some(update) = terminal.update {
if !hir.blocks.contains(&update) {
^^^^^^^^^^ borrowed immutably here
terminal.update = None;
^^^^^^^^ mutable borrow still active here (and also bc of the loop)
}
}
}
}
```
I quickly worked around this as we do in the other passes here by first building
a set of the block ids contained in the function (so that the
`hir.blocks.contains()` call becomes a call to the copied set of block ids).
An alternative would be to add a helper function for mutable iteration which
takes the desired value out of the data structure so that you can safely mutate
it and reference the rest of the HIR. Usage might look like this:
```rust
hir.blocks.each_mut(|mut block, hir| {
if let TerminalValue::ForTerminal(terminal) = &mut block.terminal.value {
if let Some(update) = terminal.update {
if !hir.blocks.contains(&update) {
terminal.update = None;
}
}
}
})
```
The lambda would receive the current `block` as a mutable reference, and for the
duration of the call `hir.blocks[block.id]` would be set to a sentinel value
that would crash if accessed. Meanwhile, `hir` would be a readonly reference to
the HIR, allowing the lambda to otherwise lookup information on the HIR but not
mutate it. This seems...kinda fine? But also not immediately necessary as
there's an easy and efficient-enough-workaround for the cases i've encountered
so far.
This is an initial translation of HIR (minus the ReactiveFunction bits). It's
mostly a straightforward translation. A few differences:
* Instead of Effect having an Unknown variant, we type `Place.effect:
Option<Effect>`. Maybe i'll revert that but it seems right.
* Terminal is divided into `Terminal { id: InstructionId, value: TerminalValue
}` and `enum TerminalValue { ... }`, so that we can reference the terminal's id
without caring about which kind of terminal it is.
* Each instruction value variant gets a named struct, whereas in JS we had lots
of anonymous InstructionValue variants.
* Rust generators are still an unstable nightly feature and likely to change
syntax enough that it seems safer not to rely on them. So
`eachTerminalSuccessor()` returns a `Vec<BlockId>`. If the perf of this is bad
enough we can switch to something like SmallVec (a popular crate) which stores a
few elements inline on the stack to optimize for small vecs (which our case
should be).
Perhaps more importantly, what's _not different_: I kept the existing approach
of BasicBlocks having a Vec of owned instruction instances, with
InstructionValue variant operands as Places rather than indices into a shared
instruction array. If this works out it will make porting straightforward.
However there's one aspect of this that is incorrect: right now each `Place`
owns its `Identifier`, whereas in JS the Identifier instances are shared mutable
references. I'll definitely need to refactor the data structures to allow
sharing Identifiers in some form, at which point we may want to change other
things too.
Start of an `estree` crate for representing ESTree with serialization to/from
ESTree JSON. To get the serialization to work quickly I took some shortcuts and
uses a slightly less precise modeling. Longer-term we can write some custom
serialization code and get more precise types (eg avoid the `ExpressionLike`
enum in favor of proper Expression and ExpressionOrFoo types).
lowerIdentifierForAssignment does extra checks such as checking if globals are
lvalues.
UpdateExpression lowers into an assignment and previously we missed out on such
checks.
I incorrectly included these as deps, we were only using these to verify
codegen. It seems fine to leave in but when I imported it internally vscode
would error, so just remove it
- Made most static methods on CompilerError take a single options object as an
argument. With the exception of invariant which takes a condition and an options
object. - Added a new `suggestions` field on CompilerErrorDetail, which we'll
use to provide eslint suggestions - Updated eslint-plugin-react-forget to handle
suggestions
Most of these errors were incorrectly using InvalidInput as a catchall for
rejecting code. I went through each one and manually updated them to be more
accurate
Found this while running Forget on the React tests.
This isn't a high priority because the ESLint plugin would've caught this. But
it'd be nice if either our validation rules caught this or if our compiler did
correctly eliminate the dep array.
This allows us to mimic the `invariant` api, which means you can just assert
that something holds true after execution proceeds to the next line without
throwing
These were effectively unused since we almost always passed in null when
creating errors, so remove them and instead pass in a `loc` explicitly. The
downside is that we no longer will see Babel codeframes in our test fixtures,
which doesn't seem like a huge loss
Fixes the mutable range validation in PrintHIR: when we checked the identifier
scope's range we failed to allow end=start=0 cases which are also valid.
While I was here, I created a separate assertion pass to check all ranges, that
way we aren't relying on the printer to validate them. The pass is on for
playground and tests, but disabled by default so it can't break internal apps.
Distinguishes between two types of "validation" passes:
- Passes which assert the validity of the HIR. These remain in HIR/ but are
renamed "assertFoo".
- Passes which validate that the user code is correct. These move to
Validation/.
Fixes#1751. Function ids can be plain strings, and we can refer to them as
globals rather than via a local identifier. In addition to the bug from $1751
this also cleans up an existing todo.
This is a fix specifically for the `enableOptimizeFunctionExpression` feature
(disabled by default). I tried running all of our fixtures with that flag on
everywhere, and this was the only issue. It's actually extracted from another
fixture which is more complicated, this is a distilled version. There were two
bugs:
* DCE was running after LeaveSSA when it needs to run before. Fixing the order
means code inside the function expression stops getting removed.
* In the feature flag, context variables share Identifier instances with the
surrounding code, whereas they are distinct instances without the feature. This
meant that mutable ranges from inside the function propagated outside the
function, throwing off our inference. The fix was to reset the ranges of context
variables after inferring the function expression's effects.
inference
Integrates the new inference to detect definitive mutations of context
variables. This allows us to more precisely infer mutable function expressions
and validate that they aren't passed where frozen values are expected.
New pass to infer context variables which are definitively mutated,
distinguishing from context variables which _may_ have been mutated. See the
code comment on the new pass for more details.
Small tweak necessary for the subsequent PRs, refs and ref values take
precedence over other mutations when computing the effect type of context
variables. We always want to record ref access within a function as capture
(since we have later validation) rather than a mutation. For now this has no
impact, either order records a Capture. But it allows later diffs to make
non-ref cases have other effects.
The original version of the code wasn't checking return values. I missed this
since my examples were passing functions _into_ the return value, as opposed to
return the functions directly. This revealed some existing test fixtures that
were technically invalid, but easy to fix by changing the return value.
In InferReferenceEffects, locations that are ConditionallyMutate are either
recorded as a Mutate or Read, which means we lose the distinction btw
conditional/unconditional mutation in later passes. This PR changes to remember
that these places were conditionally mutable, used in later analysis.
Defaults to false, ie it runs the codegen pass. When enabled it will simply run
all passes up to codegen and then skip over it. Naming of this option is copied
from [TypeScript](https://www.typescriptlang.org/tsconfig#noEmit) which has the
same named option that makes the compiler only perform typechecking.
I'm adding this option primarily to get around some issues running the eslint
plugin on Meta code. The plugin would error because Forget would report
duplicate Babel AST nodes, which I presume would only occur during codegen. It
should also make it a tiny bit faster to not run codegen, which is a nice plus.
This was causing some issues in the eslint plugin where the babel `hub` wasn't
defined. afaik the hub is only setup when running the plugin as part of a babel
pipeline, instead of a manual parse/traversal. We're using some of that infra
for printing codeframes
The existing errors thrown were marked as InvalidInput, which is now considered
critical. This was causing an error in the sync since we had 2 occurrences of
the errors being thrown. These are really todos and not invalid code.
Turning off this flag makes only critical errors throw, so TODO errors will no
longer be surfaced by the plugin. The previously failing test for unsupported
syntax is now valid.
We were incorrectly calculating the dependencies of nested lambdas, because the
"component" scope was incorrectly set to the next closest parent rather than
outermost React function (component/hook) being compiled.
This PR adds a new feature which enables additional validation/optimization of
function expressions, gated by the `enableOptimizeFunctionExpressions` feature
flag. When disabled, we actually revert the changes earlier in this stack, and
do all our lowering of function expressions in AnalyzeFunctions.
When the feature is enabled, we incrementally process function expressions in
the various compilation stages, eg InferTypes infers into function expressions,
ConstantPropagation propagates constants into function expressions, etc. Because
this stage optimizes function expressions, in this mode codegen uses the HIR as
the source rather than the original babel node.
The feature is disabled by default so it has no impact on generated code. For
now i've enabled the feature on just one test to demonstrate constant
propagation into a function expression.
Updates InferTypes to perform type inference across function boundaries.
Specifically InferTypes is now responsible for driving type inference of
function expressions (rather than deferring to AnalyzeFunctions to infer
functions), and type inference now traverses into function expressions and
infers types of free variables taking into account information from the outer
context. This relies on the fact that identifier ids are consistent across
function expression boundaries and that all free variables in functions are
guaranteed to be effectively `const`, since we promote non-const variables used
in function expressions to context variables.
Currently the process of lowering function expression bodies is deferred until
AnalyzeFunctions. However, per the motivation of the previous PR, we'd like to
be able to perform inference across function expression boundaries. To do that
we need to use consistent identifier ids across function expression boundaries.
This requires SSA conversion of function expressions at the same time as we
enter SSA.
To start, this PR moves MergeConsecutiveBlocks for function expressions into
that phase.
Generally we should reserve the `packages` directory for packages that
are needed to run the compiler, while `apps` is for frontends that might
make use of the compiler (and optionally, related packages)
This PR updates the conditions for which variables are promoted to context
variables.
The previous rule was to promote any variable where the variable was reassigned
in some function expression other than the function that declared the variable.
Notably, this meant that we did not use context variables for variables which
were captured in a function expression, but reassigned _outside_ a function
expression.
The new rule is more consistent: we promote any variable which is a) reassigned
somewhere and b) referenced in some function expression outside of their
declaring function. The implementation builds two sets of identifiers, one for
each criteria, then takes the union of these two sets.
## Motivation
The motivation for this change is to unblock additional validations and
optimizations of function expressions. It's currently difficult to translate
metadata that we infer about identifiers outside of a function expression into
metadata about the identifiers within a function expression — for example to
infer types within function expression bodies based on type information outside,
propagate constants into functions, infer reference effects, etc.
After this change, the only free variables inside function expressions will be
variables that are effectively `const` - never reassigned anywhere. Thus it will
be safe to renumber those identifiers to match the outer context (during
EnterSSA), making it trivial to map metadata from outside the function into the
function.
This change also more closely models the runtime representation — any variable
referenced in a function, and reassigned somewhere, would have to be compiled
(ie in a JS engine) to use a context variable.
The goal of this PR is to improve ValidateNoRefAccessInRender to find function
expressions which a) access refs and b) may be called during render. Currently
we always allow ref access in any function expression, but that's obviously
optimistic.
For the approach, the observation is that we already have a system that tells us
whether a function may get called — mutable range inference. So long as we
consider a function "mutable", we'll infer a range for it, but the problem is
that we don't currently view functions which depend on refs to be mutable. So
here I'm doing ~~sort of~~ a hack to force function deps on refs to be treated
as Effect.Capture. This is enough for the function to be considered mutable, for
a mutable range to be assigned, and for us to detect that during ref validation.
I don't love the hack, i'm open to other ideas!
---
- [patch] Find context variables within FunctionDeclarations (previously
missing)
- [todo] Need to fix non-allocating values / variables being DCE'd
One approach is to label everything referenced by a lambda as context variables.
I couldn't come up with other examples that break without this change, so I
wonder if something lighter / hackier works just as well. (My only hesitation is
that we may end up losing out on potential optimizations for everything aliased
to these variables).
---
(wip, waiting for feedback on workplace post)
- remove calls to `isInROMode`, as we want to log all mutations after 'freezing'
a value (both within and outside of render cycle)
- add `source` parameter -- this is the function name of the parent component /
hook
- Gating checks for debugging / profiling can be moved to within the
instrumentation or makeReadOnly functions. This simplifies codegen and reduces
code bloat.
- Warn if importing conflicting identifiers
- Aggregate imports from the same source
---
Emit calls to makeReadOnly for memoized values.
```js
function MyComponent() {
let x;
if (c_0) {
x = // ... (recompute x)
$[0] = __DEV__ ? makeReadOnly(x, "MyComponent") : x;
} else {
x = $[0]
}
}
```
- import source / specifier should be configurable, as
- we'll likely want to add gk gating to `makeReadOnly` itself to reduce codesize
bloat
- each Forget project needs different logging and filter configurations
- codegen function name as an argument for easier debugging
- only freeze memoized outputs
This is an attempt to scaffold yarn workspaces into our repo with as minimal as
possible changes to our current setup and directory structure. Essentially this
PR moves current `src` into `packages/babel-plugin-react-forget` as a first step
to keep everything working. Later on when we're ready we can split out a
`react-compiler` package that is decoupled from Babel, but it's too early for
that now
Not to get ahead of myself (sorry i had to), but i think this is the last order
of evaluation bug. At least it's the last one we know of[1]. Per the previous
PR, the issue is that constant propagation can copy the last value of a sequence
expression to where the sequence is used, leaving the original sequence
expression out of order after other instructions are moved around. We fix that
here by explicitly skipping constant propagation for the last value of a
sequence block.
[1] There are some places where we _would_ have evaluation order bugs if we
allowed arbitrary expressions, but we explicitly limit the expressions we allow
in those places. For the curious: switch test case values and destructuring
default values.
Changes the lowering for sequence expressions to use the new terminal. When
converting to a ReactiveFunction, we convert these terminals into
ReactiveSequenceValues, which nests the instructions and preserves order of
evaluation in the output.
The only catch is constant propagation — constant propagation breaks
order-of-evaluation because it can effectively copy the final value of a
sequence elsewhere, leaving the original sequence in the wrong place. I'll
address that in a follow-up.
I realized that we can use our value block system to fix _most_ of the remaining
order-of-evaluation issues we had with sequence expressions. This PR adds a new
SequenceTerminal to HIR; there is already a ReactiveFunction equivalent
(ReactiveSequenceValue) that the next PR will convert this terminal into.
Handles three more cases:
* Template literals
* deletion (property/computed)
* type casts
Only the latter has an observable impact, though i added tests for deletion just
in case and found a bug. For type casts, they're reasonably common internally
for fixmes, so this PR will help to ensure we don't drop type information just
because of a cast.
Renames some error fixtures for clarity, "error.invalid-*" are fixtures that are
expected to fail for invalid input, where other "error.*" fixtures are basically
todos. While i was here i clarified the error messages for invalid useMemo
callbacks, and changes the error severity from Invariant to InvalidInput.
Fixes one more category of bug. For assignment expressions, we validating
against redeclaring a global variable when the assignment target was an
identifier, but not when the global was reassigned via destructuring. This PR
adds a `lowerIdentifierForAssignment()` helper and uses it for assignment of all
identifier variants, including destructuring.
I reviewed the test cases we have marked as bugs ("_bug.*") and realized that
several of them are already fixed — woohoo! Then one wasn't fixed _yet_: our
type inference loses track of refs if you stash them inside an object/array. But
that's why I added the ValidateNoRefAccessInRender pass, which i've updated to
detect and reject these invalid cases. There are now only a few bugs left (more
fixes coming).
These are still quite Babel specific, but the interface is slightly more
generic: CompilerEntrypoint takes a non-Babel specific CompilerPass as an
argument instead of passing Babel's PluginPass directly.
In the future we can consider lowering the whole Program into HIR but that
involves a significant lift in our representation and a small amount of new
syntax to support (eg import statements), so this PR is the extent of this stack
for now
Updates PruneNonReactiveDependencies to treat setState functions as
non-reactive, since we know they have a stable identity. This is based on type
inference and our recently added definitions for useState and its return type,
so it's conservative and will only work when our inference can prove that the
scope dependency has the SetState type.
Note that this approach is simple and has limitations, notably the fact that the
setState is non-reactive doesn't propagate. But it's simple, trivially correct,
and already improves codegen somewhat, so i figured it's worth landing for now.
## Test Plan
Tested on internal app
Adds validation to reject freezing mutable lambdas, since lambdas cannot _be_
frozen, they're either frozen or not. Example invalid code:
```javascript
function Component(props) {
const x = {value: ""};
const onChange = (e) => {
// MUTATION!!!!!
x.value = e.target.value;
setX(x);
};
return <input value={x.value} onChange={onChange} />;
}
```
Note that there is a separate issue in which we are not detecting lambdas that
would definitely modify immutable values. We may need to distinguish
ConditionallyCapture (captures if the value is mutable, otherwise readonly) from
Capture (definitely mutates) in order to make that case work. But already this
validation helps prevent some invalid code.
Adds a `removeAllMemoization` flag that runs the entire compiler pipeline but
strips out all memoization. The intent is to be able to compare (in limited
use-cases) the performance of an existing app with all memoization removed, vs
the performance with manual memoization, vs the performance with Forget enabled.
In terms of how this works: we already strip out useMemo/useCallback since
Forget is more accurate. The new option adds an extra pass that strips out all
reactive scopes. Collectively this leaves ~zero memoization within components
(this does leave React.memo, but close enough).
---
Remove jest fixture tests in favor of snap runner. Main reasons:
- maintaining feature flags and compatible behavior required syncing all changes
to 3 files (`generateTestsFromFixtures`, `compiler-test`, and `compiler-worker`)
- jest snapshot test file causes rebase conflicts on most rebases
- speed 🙌
$ time yarn test compiler-test
(the extra test here is `has a consistent extension for input fixtures`)
```
Test Suites: 1 passed, 1 total
Tests: 37 skipped, 480 passed, 517 total
Snapshots: 479 passed, 479 total
Time: 27.668 s
Ran all test suites matching /compiler-test/i.
✨ Done in 43.18s.
yarn test compiler-test 57.05s user 3.85s system 139% cpu 43.546 total
```
$ time yarn snap
```
478 Tests, 478 Passed, 0 Failed
✨ Done in 13.12s.
yarn snap 53.96s user 9.35s system 468% cpu 13.518 total
```
Jest and snap should have the same set of features:
- report test failures via exit status (used by Git Actions)
- watch mode
- breakpoints + `debugger` statements
- note that `--sync` is not required for this
- skip `todo.` prefixed fixtures
- fixtures in nested directories e.g. `rules-of-hooks/testname.js`
- filter mode (via editing `testfilter.txt`)
- filter + debug mode
(1) edit `testfilter.txt` to filter out all but one test
(2) add `@debug` pragma to the first line of the test
testfilter.txt
```js
// @only
testfixture_basename1
testfixture_basename2
```
Turns out I just forgot to forward `process.env` when forking 😅
`debugger` statements and breakpoints should work with both `--sync` and
`--no-sync` (default) modes
This is the example we discussed in our design sync.
```javascript
function Component(props) {
const [x, setX] = useState({ value: "" });
const onChange = (e) => {
// INVALID! should use copy-on-write and pass the new value
x.value = e.target.value;
setX(x);
};
return <input value={x.value} onChange={onChange} />;
}
```
Here `onChange` is a mutable lambda, and it should be invalid to pass a mutable
lambda where a frozen value is expected. This is because unlike other value
types, you cannot freeze a lambda — the only choice is to not call it at all.
Note that there is a harder case to catch:
```js
function Component(props) {
const [x, setX] = useState({ value: "" });
const onChange = (e) => {
// INVALID! should use copy-on-write and pass the new value
x.value = e.target.value;
setX(x);
};
const x = constructAValueThatMaybeAliasesItsInput(onChange);
return <input value={x.value} onChange={x.maybeGetTheLambdaBack()} />;
}
```
This case demonstrates how mutable lambdas can be captured and then accessed
later — the analysis to catch this case is more sophisticated bc it involves
inferring that `x` aliases a mutable lambda. But we also can't be sure that `x`
does alias the lambda, so disallowing this code could prevent a lot of valid
code from compiling. My hypothesis is that we should start with at least
validating the example at the top, while allowing the second case for now.
We can now type `Array.prototype.{map,filter}`:
* The callee is ConditionallyMutable because, although the array itself is not
modified, its items flow into the lambda and may be modified there.
* The argument is ConditionallyMutable because it accepts both mutable and
immutable lambdas. Mutate would disallow immutable lambdas (wrong), while Read
would be incorrect for mutable lambdas since calling them triggers mutation.
Adds test cases to ensure we're correctly inferring mutative builtin operations
— property store, computed property store, property deletion, and computed
property deletion — as definite mutation and that we're rejecting inputs where
these operations are used on immutable/frozen values.
Adds back `Effect.Mutate`, and changes so that `Effect.ConditionallyMutate`
never rejects frozen/immutable values, while `Effect.Mutate` _always_ rejects
frozen/immutable values.
We currently use `Effect.Mutate` both for places that _may_ mutate (ie untyped
function calls) and for places that have known mutation (typed function calls,
or operations like `delete x.y`). We then use a separate mechanism to decide
whether to reject the input, with some call paths checking the effect and others
not.
This stack refactors this logic in InferReferenceEffects per our discussion, so
that `Effect.ConditionallyMutate` is for "may or may not mutate" either because
we're not 100% sure (untyped function) or because the mutation depends on the
operand (ie, a callback arg that will be invoked and thus will mutate if the
lambda is mutable, not mutate if the lambda is immutable). Later diffs add back
`Effect.Mutate` as "definitely 100% mutating".
Defines 4 new types:
* Return type of `useState()`, which has properties "0" and "1" to allow us to
infer the types when destructuring
* Type of useState() set state function
* Return type of `useRef()` so we know what is a ref
* Type of ref.current, so we know what is a ref *value*
Example:
<img width="1670" alt="Screenshot 2023-05-24 at 9 59 37 AM"
src="https://github.com/facebook/react-forget/assets/6425824/3ee7d04a-fda3-4b7b-89b7-d205d9a6fd0d">
---
Toggle default to true, since this should be a no-op refactor.
Tests:
- test fixtures
- ran on Store + - and saw no difference in compiled output
- [diff](P744101621) with
`enableTreatHooksAsFunctions=false`
- [diff](P744105565) with
`enableTreatHooksAsFunctions=true`
Adds a new feature flag which tells the compiler to assume that hooks follow the
Rules of React. Specifically, the idea that since any hook could be wrapped in a
giant `useMemo()` call, all arguments to hooks have to be treated as if they're
owned by React — and therefore become immutable — and that the return value of
the hook is immutable.
Our default is to assume that hooks break the rules, but in practice nearly
every component follows them.
Updates the InferReferenceEffects logic for CallExpression to work similarly to
MethodCall, where we take into account the function signature (if present) when
inferring the effects and return kind.
Defines the `Boolean`, `String`, and `Number` global functions. This will be
useful for allowing developers to wrap statements that produce a primitive in a
way that Forget knows about in order to optimize better.
Handles some edge-cases where we previously flattened away some of the structure
of a labeled block, instead ensuring that we retain the original shape. See the
output.
## Test Plan
Tested on the internal app we're focused on (w useMemo inlining enabled), it
works fine.
I realized a wayyyy simpler approach to inlining a lambda: wrap it in a labeled
block. The transformation is roughly as follows:
```javascript
// Before
const x = useMemo(() => {
if (a) {
return b;
}
return c;
}, [a, b, c]);
return x;
// After
let x;
label: {
if (a) {
x = b;
break label;
}
x = c;
break label;
}
return x;
```
The key to making this work is fixing up some edge cases in labeled blocks,
hence the previous PRs.
This is part of a stack to fix some edge cases in inlining of useMemo closures.
In this first step, I'm disabling `shrink()` in order to retain more information
about the data flow. For example,
```
label: if (cond) {
break label;
}
return foo;
```
Would previously have shrunk away the if body, making the IfTerminal.consequent
point directly to the fallthrough block (w the return). Now we retain a separate
block.
- delete output files when we detect input files are deleted
- enable test fixtures in nested directories
- exit with error code when we detect failures
Note that the test failure on this PR is expected and will be fixed by #1608 (or
happy to abandon that PR and fold the changes)
---
Looks like we delete `.js` files but missed `.expect` files. Jest probably
didn't catch this because the basename of the fixtures had duplicates (in
rule-of-hooks)
---
(`react-forget-runtime` package seems to be synced to -.)
RFC: useRenderCounter hook:
- tracks # renders (increments on render path)
- exposes a global renderCounterRegistry (counting # renders in alive / mounted
components)
Next PR will modify BabelPlugin to add codegen
```js
// Similar to how we're currently importing `isForgetEnabled`
import {isInstrumentForgetEnabled_Secret} from "ReactForgetFeatureFlag";
// ...
function Component_uncompiled(props) {
if (isInstrumentForgetEnabled_Secret) {
useRenderCounter();
}
// ...
}
function Component_forget(props) {
if (isInstrumentForgetEnabled_Secret) {
useRenderCounter();
}
// ...
}
```
I originally created a separate test for the mode with JSX memoization disabled,
but we can merge this into the main compiler-test and enable the feature with a
pragma.
Reverts #1502, but flips test flags (e.g. `inlineUseMemo` by default, unless a
test specifies `@inlineUseMemo false`. I figured this add less thrash for test
fixtures, but happy to just do a clean revert (or remove the pragma altogether
and always pass `inlineUseMemo: true`)
---
In `lower`, we now ensure that all context variables are declared by a
`DeclareContext` instruction. `DeclareContext` always produces a `let`
declaration, and `StoreContext` is always a reassign. There are a few reasons we
need `DeclareContext`:
- DeclareLocal assumes it is storing to a SSA-fied identifier (which always
stores an immutable primitive). This does not fit context variables.
- Without DeclareContext, we need custom logic in some passes to initialize
identifier / context state (e.g. `MutableRange`, ValueKind, etc) for the
`StoreContext` that declares the context.
This PR stack models context variables as concrete identifiers (with references
to context variables modeled by `Place` referencing the context variable
identifier). @josephsavona pointed out that this is abusing the notion of
Identifier/Place, as context variables are essentially interior properties of a
ContextEnvironment. Since we are not modeling `ContextEnvironment` implicitly or
explicitly, all inference for context variables is essentially pointer analysis.
---
This PR adds LoadContext and StoreContext to handle reading and writing to
context variables.
A context variable is any variable that is declared within a Forget-compiled
function and reassigned within a closure. Conceptually, we want to treat these
variables as attributes of a `EnvironmentContext` variable (as most javascript
VMs do).
- context variables currently do not participate in type inference (i.e. we do
not produce type equations for loads from context variables). In the future, we
can try typing this as `Phi(assignment1Type, assignment2Type, ...)`.
- context variables are always treated as `Effect.Mutable`.
- context variables do not participate in SSA, or certain optimizing passes
(e.g. dead code elimination, constant propagation, etc).
There is some still follow ups:
- From my understanding, we should introduce a `DeclareContext` instruction.
- currently, declaring a context variable (without initializing it) is broken.
This is because the declaration lowers to `DeclareLocal`, which assumes it is
storing to a SSA-fied identifier.
```js
let x;
x = 4;
() => { x = {}; };
```
- DeclareContext will also make some initialization logic easier. In this PR, I
added some hack-y code to handle initializing effects / mutable ranges / other
inference state for the first StoreContext.
- Handle or bail on stores to context variables through destructuring assignment
-
~~Next PR:~~
- ~~Change closures to track reassigned identifiers (to extend mutable range of
primitives)~~
Enables hooks validation in playground. Also adds a tab to show the output of
validation (in case it passes) with the inferred post dominator tree. We can use
this to debug the dominator in case of false negatives.
<img width="1724" alt="Screenshot 2023-05-11 at 11 07 08 AM"
src="https://github.com/facebook/react-forget/assets/6425824/8f7ae472-8415-4899-aedf-c8f26094ebfe">
Incorporates the fixtures from eslint-plugin-react-hooks using a script, so that
we can easily update them in the future. For each fixture we run the compiler
with and without hooks validation first so that we know if the fixture is
expected to pass — we have some false positives and false negatives that i can
work through. For example we accidentally think that `userFetch()` is a hook,
oops. Fixtures that should pass but error, or that should error but pass, are
marked as `todo.<name>` or `todo.error.<name>`.
While i was here i added the ability to have fixtures in subdirectories for
grouping purposes.
---
Try to fix bug from #1589:
> If a declaration for an immutable identifier (i.e. one that is not later
re-assigned, since undefined is a primitive) is sandwiched between mutations, we
currently do not record it as an output or hoist it out of the reactive scope.
One simple fix is to add all declared (and later referenced) identifiers as
declarations of a reactive scope. This has some undesired effects (e.g.
additional instructions + memo cache slots), but in practice, this shouldn't be
happening often.
Alternatively, we could 1.) add a pass to hoist declarations, 2.) account for
this in constant propagation, or 3.) add a bailout
---
Record incorrect output.
If a declaration for an immutable identifier (i.e. one that is not later
re-assigned, since `undefined` is a primitive) is sandwiched between mutations,
we currently do not record it as an output or hoist it out of the reactive
scope.
While optimizing per @josephsavona's suggestions in #1592, I noticed that we
were clearing quite a few require cache entries.
As of this PR, `Object.keys(require.cache)` holds
- 1258 entries total
- 67 files compiled from Forget source code (this is what
`ts.createWatchCompilerHost` modifies)
- 1120 babel source files (from node_modules)
When working on watch mode, I'm almost always making changes to Forget source or
test fixture files. It's a bit faster to just clear those entries (assuming that
babel has no global state we need to invalidate).
On my computer, re-running tests in watch mode (triggered by source code
changes) takes:
| | All tests | One test (filter) |
|-- |--------|----------|
| current | 4.7s | 1.8s |
| this PR | 1.8s | 0.1s |
---
Some typed functions need to annotate callees or arguments as `Effect.Store`.
This PR modifies alias analysis (`InferAliasForStores`) to account for this
Snap currently has a bug in which the require cache is not correctly cleared
when running in filter mode (#tests < 2 * #workers).
- We're currently clearing all entries in the require cache of worker threads,
including `jest-worker` and `snap/dist/...` files
- jest-worker seems to `require` these files on every dispatch (i.e.
`worker.compile` seems to call `require(`compiler-worker`).compile`)
I noticed some instances of this error when running forget on an internal
product. I previously fixed the case if a logical/conditional used only for side
effects (not assigned to a variable) but the new cases were assigned to an
unused variable. I double-checked and we’ve actually fixed all the steps after
these invariants so we can just remove them and support these cases.
When we calculate the dependencies of a FunctionExpression we were only adding
new items if the binding identifier had not been seen yet. That is correct for
`capturedIds` since its the set of identifiers, but incorrect for `capturedRefs`
since its an array of all the distinct places. This meant that if a function
expression referenced multiple properties of the same binding, we'd only record
the first one. We now correctly record all of them.
Tidies up the implementation a bit, splitting the single function and class into
distinct computeDominatorTree() and computePostDominatorTree() functions and
helper classes.
React will retry or abort components that throw (depending on a few conditions),
so from React's perspective a `throw` statement is not a normal exit node. Thus
the Rules of Hooks really have a caveat: the set of hooks that are called _in an
execution that returns successfully_ must be consistent. Examples such as the
following are therefore allowed:
```javascript
function Component(props) {
if (props.cond) {
throw new Error(...);
}
useHook();
}
```
By modeling `throw` as an exit node, we rejected cases such as this. This diff
changes to not model throws as exit nodes. #1584 changes this to make it an
option, since some cases will want to consider throw as an exit node.
Incorporates the fixtures from eslint-plugin-react-hooks using a script, so that
we can easily update them in the future. For each fixture we run the compiler
with and without hooks validation first so that we know if the fixture is
expected to pass — we have some false positives and false negatives that i can
work through. For example we accidentally think that `userFetch()` is a hook,
oops. Fixtures that should pass but error, or that should error but pass, are
marked as `todo.<name>` or `todo.error.<name>`.
While i was here i added the ability to have fixtures in subdirectories for
grouping purposes.
See the code comments for more, but the basic idea here is that we use the post
dominator tree to find the set of basic blocks which are guaranteed reachable in
each function. Those are the only blocks where it is safe to call hooks, and we
error for hook calls in any other blocks.
Implements an efficient algorithm for computing the dominator (or post
dominator) tree of a CFG, following
https://www.cs.rice.edu/~keith/Embed/dom.pdf. This is used/tested in the next PR
to validate that hooks are called unconditionally.
note: I clean up the implementation quite a bit late in the stack in #1584
I noticed this while demoing Forget to React Org alum Christoph Nakazawa — in
array.map calls (and other APIs that take a lambda as input) we sometimes end up
memoizing the lambda. It's technically correct since the function _could_ return
the lambda, and then we'd need it to be memoized. It's tricky because array.map
is often called on nested objects, where even if we had type inference on the
outer value we wouldn't know for sure that the inner property is an Array and
not some other data type with a custom .map. For example in
`data.feedback.comments.edges.map(edge => ...)`, even if we knew that `data` was
an Object, we wouldn't know that data.feedback.comments.edges is an Array
without cross-file type knowledge.
But it's definitely wasteful to memoize these lambdas, so we should brainstorm
options. One option that stands out right away: if the lambda has zero
dependencies, then we could lift it out to module scope and refer to it by name.
Adds a validation pass to check that the only thing you can do with hooks is
call them. A follow-up PR (still early WIP) will check the other aspect of the
rules of hooks, that they are not called conditionally. That's a more involved
algorithm.
We previously disallowed OptionalMemberExpression inside a normal
MemberExpression, eg `(a?.b).c`. The new representation handles this case
correctly so we can remove the restriction.
Our previous lowering for OptionalMemberExpression reordered the evaluation of
properties, such that we had to restrict the allowed properties to those that
were safe for reordering. With the new representation we preserve order of
evaluation, so we can relax the restriction. This unblocks a few cases in an
internal product.
Now that _all_ optional expression types use the new representation, the
optionality of all PropertyLoad and ComputedLoad is modeled via control flow (in
HIR) and the structure of OptionalExpression (in ReactiveFunction). Thus we no
longer need the `optional` properties on these load instructions — they're
optional if they're part of an OptionalExpression.
Earlier PRs in the stack change the way we lower OptionalMemberExpression, but
only when they ultimately appear inside some OptionalCallExpression. This PR
ensures that _all_ OptionalMemberExpressions get the new lowering. Note that one
test case has what is arguably a regression, but the new behavior is also
reasonable: if we see both `a.b?.c` and `a.b.c.` as dependencies of a scope, we
previously inferred `a.b.c` as the dependency, but we now infer `a.b` as the
dependency. This isn't as optimal as what we had before, but it also seems good
enough for now. Also note that some cases are improved: `foo(a.b?.c)` would
previously have taken `a.b` as a dependency, we now take the full value of
`a.b?.c` as a dependency - more precise.
So overall i'm inclined to land and follow-up on the one regression, since the
overall model is more cohesive.
call
When we traverse an OptionalExpression in PropagateScopeDependencies, we
previously considered the entire value to be optional. With the changes in this
stack to more accurately model OptionalMemberExpression, the `object` portion of
an OptionalMemberExpression is now evaluated within the OptionalExpression. This
PR refines the handling of OptionalExpression accordingly, so that we only treat
the optional portion as conditional.
The previous OptionalCall terminal and reactive value kinds are now used not
just for optional calls, but for optional member expressions that appear within
an optional call. This PR renames those data types to OptionalTerminal and
OptionalExpression for clarity.
Extends the new modeling of the previous diff to OptionalMemberExpression. In an
example such as `a?.b?.c`, we now model that not only is the `.c` conditional,
but our control flow graph accurately reflects the fact that the `.c` is only
evaluated if `a.b` exists. Previously we knew it was conditional but the CFG
allowed a path from a being null through to evaluation of `.c`.
More accurately models nested OptionalCallExpression. Consider:
```javascript
a?.(b)?.(c)
```
Our previous representation modeled it such that we treated the second function
call as if it would be called regardless of whether `a` existed or not. We knew
that the second call was conditional, so our test output was correct, but the
control-flow graph didn't faithfully model the semantics. That bothered me.
The new representation correctly models the control flow, and the fact that if
`a` is null/undefined execution immediately aborts (not reaching the second call
at all, nor the evaluation of its args), and evaluates the whole outer
OptionalCallExpression to `undefined`.
Note that nested optional member expressions still have the previous model —
that's next to address.
A common idiom is to map over some possibly-missing list of items from a data
payload and fall back to an empty array:
```javascript
const renderedItems = data?.items?.map(renderItem) ?? [];
```
The way we were lowering OptionalCallExpression meant that in this case, we'd
end up with an OptionalCallTerminal as the terminal of the logical expression's
test block, which violates our internal invariant. Logical test blocks must end
in a Branch! This PR fixes the immediate issue, which is that the callee - in
this case `data?.items?.map` — was being lowered prior to the
OptionalCallTerminal instead of inside its test block. Changing that fixes the
shape of the IR and makes this example work.
As part of investigating this I realized that the way I originally handled
lowering of optional call isn't quite right. The difference isn't observable
unless we did more sophisticated DCE but we don't correctly model the fact that
if `data.items` is null that the `map()` call won't occur. That is technically
fine bc we do model the fact that the `map()` call is conditional, and notably
its arguments are only conditional dependencies. So it's good enough. But in a
follow-up I'll change to model the fact that `data.items` is null, that the map
call isn't reachable at all.
Fix the previous bug — this was a simple oversight, where FlattenScopesWithHooks
overrode `visitValue()` but failed to call `traverseValue()`. This meant that
when we reached compound expressions such as LogicalExpressions that we didn't
traverse into their nested values, and didn't see the hooks hidden there.
Repro of a bug in which we incorrect memoize hook calls that are inside logical
expressions (though the bug could occur for ternaries, optional calls, and
sequence expressions too).
This is an attempt to get down some of the principles and goals that we've had
partially written down, partially just thoroughly discussed amongst the team.
It's rough draft quality but better than nothing, and gives us someplace to add
to.
This can't be tested yet - we only support simple, safely re-orderable values as
case test values - but it will easy to overlook later so i'm adding now.
PropagateScopeDependencies is one of the few places we don't use the new visitor
infra for traversing ReactiveFunction. Or rather it _was_!
Note that there's a bit less value here than in other places since we have to
handle each terminal variant with custom logic, but at least it's more
consistent with the rest of the codebase now.
We previously didn't support ternaries whose value was unused, so we had an
extraneous temporary and console.log call to ensure the value counted as used.
We now special-case ternary/conditional expressions which are in an
ExpressionStatement to not prune them, so the temporary and log are now
unnecessary.
It turns out the third parameter to `cloneNode` is ["If the third parameter is
true, the cloned nodes exclude location
properties."](c060e5e3d5/packages/babel-types/src/clone/cloneNode.ts (L35-L39))
strips away locations if its true, so to fix simply change this to false
Defines common `console` methods to tell the compiler that they take readonly
args. This ensures that things like `console.log()` aren't accidentally viewed
as a mutation. Previously the pattern of "build object, then log it after
mutation is done" would have grouped the console.log as part of the mutation and
the log only would fire if the value got reconstructed. Now we know the log
isn't mutating, and the log will happen regardless of whether the value is
rebuilt or cached.
Updates two points in the compiler that were easy to miss when adding new
terminals:
* HIRBuilder's `removeUnreachableFallthroughs()` nulls out unreachable
fallthroughs, but this had a non-exhaustive `if` statement. It now uses a helper
function which internally has an exhaustive switch.
* LeaveSSA needs to schedule block fallthroughs, but had a non-exhaustive `if`
statement. It also uses a helper function which internally has an exhaustive
switch.
cc @poteto since you ran into this (ie the compiler not alerting you to update
these places) w your diffs.
Discovered this in a recent attempt at syncing Forget to Meta, it seems
that calling path.stop() is unsafe as it appears to have strange
behavior in plugins that come after. This resulted in `import type
{...}` not being compiled away in the post-babel output which isn't
valid JS syntax. Removing the `stop()` calls fixes it
Test plan: made these changes locally, synced my local changes to Meta and reran
- in simulator and observe that it now runs and doesn't throw a syntax error
This is a more general version of the change from #1521. That PR ensured that
LoadLocal temporaries accessed outside the instruction's scope are correctly
promoted. However, we have a similar pattern with PropertyLoad.
This PR adds a general mechanism for handling these type of indirections: any
LoadLocal/PropertyLoad temporary accessed when it's defining scope is not active
will be promoted to a declaration of the defining scope. Notably, we do this in
a way that ensures that the dependencies are preserved, ie that we correctly
view the operand of LoadLocal/PropertyLoad as a dependency of the current scope.
Supports EmptyStatement nodes by ignoring them. Note no tests because
format-on-save clears away any empty statements and there is, rather
frustratingly, no way to tell VSCode not to format on save for a specific file
via the file contents itself or project configuration (at least, not that i can
find).
Uses the new ExpressionStatement instruction to ensure that logical and
conditional expressions are never pruned. This addresses an issue where we were
unable to construct a ReactiveFunction for unused logical/conditional bc there
wasn't a single Identifier assigned in both branches. The ExpressionStatement
ensures that the result is used, that we don't prune the phi, and that both
branches have a single assignment target.
In theory we could be more sophisticated with DCE and still prune these
instructions if their operands are also safe to prune, but in practice you're
only like to have a logical/conditional as an expression statement (in the
source) if it's for side effects.
Adds an `ExpressionStatement` instruction variant to model values that are
otherwise "unused" but which we don't want to remove. The next diff changes
BuildHIR to use this where appropriate.
We previously represented JsxExpressions using builtin tags - `<div>`, `<b>` etc
- by lowering the tag name to a Primitive with the string name of the tag.
However, by lowering into an independent value, it was possible that the lowered
tag name could be grouped into a different memo slot, such that we ended up with
output like:
```javascript
let t0;
if (c_1) {
...
t0 = "div"
...
} else { ... }
return <t0>{children}</t0>
```
This is obviously wrong. It's also wrong to rename `t0` -> `T0`, because React
treats that as a custom component, not a builtin. The right thing is to
explicitly model builtin components, which this PR does by making
`JsxExpression.tag` be a union of Place | BuiltinTag.
Ensures that temporaries used in JsxExpression tags are named with a capital
letter so that they are treated as custom components rather than builtins.
---
Changes:
- Added `testfilter.txt`
```
// @only
call
capture-param-mutate
jsx-spread
```
or
```
// @skip
call
error.todo-kitchensink
```
- grouped all commands under `--mode`
```js
// runs all tests
yarn snap
// runs all tests and updates fixtures
yarn snap --mode update
// runs only tests that pass `testfilter.txt`
yarn snap --mode filter
// run in watch mode
yarn snap --mode watch
```
- in watch mode, toggle between running all tests or filtered tests
```
386 Tests, 386 Passed, 0 Failed
Completed in 4994 ms
Current mode = NORMAL, run all test fixtures.
Waiting for input or file changes...
u - update all fixtures
f - toggle (turn on) filter mode
q - quit
[any] - rerun tests
> f
PASS call
PASS capture_mutate-across-fns
PASS timers
3 Tests, 3 Passed, 0 Failed
Completed in 39 ms
Current mode = FILTER, filter test fixtures by "testfilter.txt"
Waiting for input or file changes...
u - update all fixtures
f - toggle (turn off) filter mode
q - quit
[any] - rerun tests
```
---
- `runner.ts` is pretty large now, happy to split it up into multiple files
- I'd also like to refactor `watch` to make its shared state and control flow
explicit
A bit of a hack -
We currently trigger test runs when we detect changes in the test fixtures
directory. This trigger is also hit when we run `snap` in update mode, since
updating performs file writes.
This PR will ignore subscription changes (callbacks) that trigger within 5
seconds of the last update.
It seems difficult to be more granular with a timestamp, since `@parcel/watcher`
doesn't give us the file change timestamp and (from my understanding), other
promises and tasks can be queued to run between the update and callback.
Run snap tester on a forked node process so runs can be interrupted. (Currently,
`Ctrl+C` is not handled until after all test fixtures finish compiling). This
*feels* a bit heavy-handed, but main.ts is pretty small and doesn't do much
other than listen for input / signals. Would love feedback here since I haven't
really worked with nodejs / JS cli tools before
- main.ts
new file that spawns forked runner
- pipe stdin/out/err to and from the child runner process, details described in
comments.
- listens for exit event of child
- runner.ts
- added logic to listen for interrupts and clean up (not really familiar with
how file and tsc watchers are implemented, so we try to call 'close' on them
just in case they need to release locks / do other cleanup)
---
`yarn snap --sync` currently fails on `error.file-has-non-critical-errors`. This
is because we're relying on a globally overwritten `console.error` function to
report non-fatal errors. However, executing `Promise.all(...)` on a single
nodejs thread will interleave calls to `run` (which is an async function).
---
Next PRs: skip / only tests, pretty diffing
This PR:
1. Add help messages:
```
$ node packages/snap/dist/runner.js --help
Options:
--version Show version number [boolean]
--sync Run compiler in main thread (instead of using worker threads
or subprocesses). Defaults to false.
[boolean] [default: true]
--worker-threads Run compiler in worker threads (instead of subprocesses).
Defaults to true. [boolean] [default: true]
--watch Run in watch mode. Defaults to false (single run).
[boolean] [default: false]
--update Run in update mode. Update mode only affects the first run,
subsequent runs (in watch mode) require typing `u` to
update. Defaults to false. [boolean] [default: false]
--help Show help [boolean]
✨ Done in 0.62s.
```
```
...
386 Tests, 386 Passed, 0 Failed
Completed in 4434 ms
Waiting for input or file changes...
u - update fixtures
q - quit
[any] - rerun tests
```
2. Surface typescript diagnostics; skip test fixtures if source code has errors
```
$ node packages/snap/dist/runner.js
src/Optimization/ConstantPropagation.ts:87:3 - error TS1434: Unexpected keyword
or identifier.
src/Optimization/ConstantPropagation.ts:87:3 - error TS2304: Cannot find name
'lt'.
src/Optimization/ConstantPropagation.ts:87:6 - error TS2552: Cannot find name
'hasChanges'. Did you mean 'onhashchange'?
src/Optimization/ConstantPropagation.ts:135:11 - error TS2552: Cannot find name
'hasChanges'. Did you mean 'onhashchange'?
src/Optimization/ConstantPropagation.ts:155:10 - error TS2552: Cannot find name
'hasChanges'. Did you mean 'onhashchange'?
Compilation failed (5 errors).
Found errors in Forget source code, skipping test fixtures.
✨ Done in 10.73s.
```
```
Compiling...
src/Optimization/ConstantPropagation.ts:86:39 - error TS2552: Cannot find name
'HIRFunctin'. Did you mean 'HIRFunction'?
Compilation failed (1 error).
Test: Found errors in Forget source code, skipping test fixtures.
Waiting for input or file changes...
u - update fixtures
q - quit
[any] - rerun tests
```
DevTools relies on built-in hook names at their call site to be unprefixed in
order to correctly track them. This PR updates our Babel plugin to:
- Check if there are any existing import declarations of `import { /* ... /* }
from 'react';`
- If true, we add the specifier `unstable_useMemoCache as useMemoCache`
- Otherwise, we synthesize a new import declaration
- In Codegen we now emit `useMemoCache(n)` rather than
`React.unstable_useMemoCache(n)`
I forgot to remove this when we removed the diagram output. This brings test
time from 6s -> 5s on my machine, still much slower than the new test runner in
#1486.
#1507 Ensured that declarations of reactive scopes were propagated to parent
reactive scopes as necessary to ensure that those declarations would be
available at the appropriate block scope. This meant that some scopes that were
previously pruned would no longer be pruned. Specifically, an outer scope wo any
declarations, but which contained a nested scope _with_ a propagated
declaration, would now end up with non-empty declarations and not be pruned.
This PR changes to track the declaring scope of each declaration, so we still
prune scopes that don't have any of their own declarations.
Originally I defined `Stack` as an interface to ensure both Node and Empty
variants would have an identical API. But exporting an interface allows a
developer to define other implementations, when we really want to ensure that a
Stack is precisely a Node or Empty instance. This PR changes to exporting a
union of `Stack = Node | Empty`, and makes the interface private to the module.
Fixed a bug identified in repro cases earlier in the stack. The case is where
some later value is composed of several values, say A and B, where A is an
identifier that is reassigned within B. Also, the mutable range of B surrounds
the evaluation of A. In this case, the reference to A gets lowered to a
temporary (say a t0 = LoadLocal A), and that temporary is created within the
reactive scope for B.
PropagateScopeDependencies bypasses LoadLocal indirections, and considers the
reference to the temporary (t0) as if it was a reference to the identifier (A).
That breaks the whole reason we lower Identifiers to temporaries - to preserve
evaluation order.
This PR fixes the bug by promoting temporaries to names values if they are
referenced outside their defining scope. So, the reference to t0 stays a
reference to t0, which correctly preserves the value of A at the right point in
time.
This is all much easier to see in the new test case.
When lowering a JSX element we were correctly lowering to a temporary in all but
one case: the common case of an identifier. That is fine in practice but breaks
in the presence of the tag identifier being reassigned in the props/children.
This PR fixes to always lower the tag to a temporary.
This PR updates ConstantPropagation to support propagating global references:
```javascript
// Before
const x = Math;
foo(x);
// After
foo(Math);
```
This is a generally useful optimization but also helps with a subset of cases
around JSX element tags, which are frequently globals.
During codegen, when we now cache and restore the temporary values map as we
enter and exit the scope. This ensures that any temporaries within the reactive
scope are only visible within that scope, and not to subsequent code. In a
subsequent PR this surfaces a bug with temporaries not correctly exported from a
reactive scope.
In codegen when we lower an operand we check to see if we have an already
lowered value for it (stored in `cx.temp`). Currently we silently handle missing
values by emitting a raw identifier, but this is really an error. This PR adds
validation, which uncovered a few places where we legitimately won't have a
value - things like function parameters that got swapped for temporaries bc of
destructuring. We populate those as `null` values now, and fail if a temporary
had a truly missing value.
Adds an option to always throw errors regardless of severity (default, ie the
status quo), or when the flag is disabled, only critical errors will be thrown.
Any error that isn't considered a critical error (see
`CompilerError.isCritical()`) since it might indicate that the compiler is
buggy, while non-critical errors will result in that file being skipped for
compilation, but otherwise continue compiling other files
* JSX tag value temporaries getting promoted due to being sandwiched inside the
mutation of some other item
* Incorrect order-of-evaluation for jsx tag relative to props/children
snapshotting
As demo'd on our sync. This is meant as a replacement for `yarn test` just for
fixtures. On my machine, a test run from a steady state of Jest watch mode takes
6 seconds. With this script, it takes ~800ms.
Workflow: make edits in `packages/snap/`, then `yarn build` in that directory to
build the test runner.
To run tests, `node packages/snap/dist/runner.js` from the main forget/
directory to run tests. Pass `--watch` for watch mode, `--update` to update
snapshots. Note that this _only_ updates .expect.md files, it does not update
Jest's `__snapshots__` directory (this is intentional, our use of Jest snapshots
is a hack that this script is meant to replace).
When running in watch mode, ctrl-c or q will quit, 'u' will update snapshots,
and any other key will re-run tests. Tests will re-run on changes to the source
code (after an incremental TS rebuild) and on changes to the fixtures.
Main things that are missing:
* Don't run tests if TS compilation had errors (the errors should be logged to
console already, but we need to wire that up so that we abort tests if there are
errors)
* Actually delete stale files in update mode (there is some commented-out code
to double-check and then uncomment)
* Improve diff view: just print the diffed segments, not the whole file diff.
See the API at https://github.com/facebook/jest/tree/main/packages/jest-diff
(right now we're using `diff()` but should probably use one of the lower-level
functions)
* Properly parse/validate args with `yargs`, add a help option, etc
* In watch mode, when tests finish print a list of commands so users know what
they can do (like Jest does)
* Support skipping tests and selecting a single test to focus. Suggestions:
* Name files with `skip.` to skip
* Allow passing a test name to the script to run only tests matching that
pattern, eg `node runner.js -p <pattern>`
* In watch mode, support typing 'p' to prompt the user for a pattern. After
that, support typing 'a' to clear the pattern and run all tests.
Fix for bug demonstrated in #1506. When we add variables as output of their
defining scope, we need to propagate this information upwards to all parent
scopes which are not current active.
Minimal(ish) repro of a bug we saw internally, where an output of a nested
reactive scope is defined at the wrong block scope, and so later references to
that value are invalid.
The simplified structure is:
```
scope0 inputs=[] outputs=[] {
scope1 inputs=[] outputs=[t0] {
t0 = ...
}
}
t0
```
Note that `t0` correctly appears as an output of the inner scope1, but not as an
output of the outer scope0. We need to propagate outputs upward as necessary to
ensure they are available at the right block scope: in this case, that would add
`t0` as an output of scope0.
An earlier version of PropagateScopeDependencies did this but it looks like it
got lost along the way (not a big deal)
This aligns the playground configuration with our internal compiler
configuration to make it easier to repro compilation issues on playground. There
is a bug that doesn't repro right now and i suspect it's because of different
hooks being configured.
Test plan:
Before: playground output has no obvious bugs, but is different than internal
compilation output where the bug occurs
After: playground output matches internal compilation output w the bug
Making ReturnTerminal.value non-nullable broke our optimization to elide final
value-less return statements. We now check if a return value is explicitly
`undefined` and elide the value in this case, which then also propagates to
allow removing the final `return` statement of a function if the value is
missing.
Updates our Babel plugin to add an import to React if we succesfully compiled a
function and cached one or more values. For now this logic is entirely in the
Babel plugin itself, but long term we should move this into codegen and teach
Forget to start the pipeline by lowering the whole Program node to an HIR (which
would also allow us to understand imports and other module scope values being
used). Right now it's not possible to add in codegen (without some ugly code) as
a file containing multiple components would result in duplicate imports being
generated
Previously useMemo inlining created a new StoreLocal assignment (not
reassignment!) instruction for every return value. This breaks when the return
is inside a block (like an if-block) as the scope is tied to the block.
For example: ``` let x = useMemo(() => { if (...) { return { ... }; } })
``` would become: ``` if (...) { const temp = { ... }; } const x = temp; ```
This PR instead changes the inlining to declare a temporary in the function
prologue and then reassign values to it when replacing return statements.
``` let x = useMemo(() => { if (...) { return { ... }; } }) ```
becomes
``` let temp; if (...) { temp = { ... }; } const x = temp; ```
This is a simplified version of #1454. The goal of this PR is to inline the
contents of `useMemo()` callbacks, rather than just immediately invoke the
lambda. Turning useMemo() into an IIFE works, but it means that we can't
optimize within the lambda block. Our investigations showed that there's a lot
of room to optimize at a finer granularity than manually written useMemo calls.
For example, one product instance had a useMemo that created a list of child JSX
elements. Most of those elements only relied on a single variable (`a`), but a
few relied on a second variable (`b). Thus _all_ elements were invalidated
whenever `b` changed. If Forget retains the original lambda, we have no choice
but to keep that (coarse) granularity for memoization. When we inline, we can
optimize to make e.g. individual JSX elements depend on their precise
dependencies.
The rough idea is:
* Keep track of all function expressions
* When we find a useMemo, lookup its function expression, and add its CFG to the
main function (the previous PR ensures that BlockIds won't collide)
* Replace any return statements with a StoreLocal to save the result and a Goto
to the code following the useMemo call.
* Then we run the usual set of passes to patch the HIR back up again.
Example:
```javascript
// Before
function Component(props) {
const x = useMemo(() => {
if (props.cond) {
return null;
}
return foo(props.x);
}, [props.x]);
return x + props.y;
}
// Intended - **before** memoization
function Component(props) {
let x;
if (props.cond) {
x = null;
} else {
x = foo(props.x);
}
return x + props.y;
}
```
This PR ensures (via a static assertion function) that all terminal variants
have a SourceLocation, and adds locations to the variants which didn't have it
before. This also adds a static assertion that terminals have an InstructionId,
though we already relied on that so it was checked via usage.
This is a prerequisite to inlining `useMemo()` lambdas so that we can better
optimize them. Nested functions are evaluated with a fresh HIRBuilder, which
means that they currently have their own `bindings` object for mapping
identifier instances to IdentifierIds. This means that identifier ids in a
closure are _always_ different that those outside the closure, even when they
refer to the same identifier:
```
function Component(props) {
props; // becomes e.g. props$1
const onClick = () => {
props // becomes e.g. props$2
};
}
```
For useMemo inlining this is problematic because we've lost the association that
these identifiers actually refer to the same thing. This PR changes that,
sharing the name resolution data structure between the top-level function and
any nested function expressions.
To unblock internal experimentation, for now let's just skip over compiling any
file that contains one or more disables of React's eslint rules, and log that.
This is a little coarse in the sense that we could skip over just functions that
contain the comments, but Babel doesn't provide an easy way to traverse comments
afaict so this is the simplest solution. I did check our internal repo and noted
that there was only one disable of exhaustive-hooks in that entire directory in
one file, so this should be fine.
Notably we are not throwing any errors if we detect these violations as we don't
want to fail the build, we just want to skip them for now.
While running the latest Forget build on www I noticed that a lot of the
bailouts were special-cases where we used interpolation in the error `reason`
string to provide more context for debugging. This is a pretty cool result,
because it means that we actually support nearly all the common syntax (at least
based on a sample of the codebase). But it makes our tools for aggregating
errors break down a bit.
This PR adds a new, nullable `description` property to CompilerErrorDetail, and
manually updates to ensure that we always pass a static `reason` and only use
interpolation in the `description`. This will allow our aggregation tools to
group by the reason.
NOTE: See background in #1476.
Updates BuildHIR to use the new LabelTerminal for LabeledStatements, and adds
support for HIR->ReactiveFunction transformation and codegen. Note that we
sometimes produce an extraneous block wrapper if it turns out the label wasn't
necessary, that seems...fine?
Adds a new `LabelTerminal` which will be used to represent LabeledStatements
that contain a statement other than a loop. What we do for these cases is
basically break the containing block in two, with a goto after the inner
statement to the fallthrough. This allows us to model the label, and any `break`
to it, in the HIR. However this fails in codegen because we can't find the
fallthrough branch — we need a high level terminal that knows about this
structure.
Hence LabelTerminal. Now, instead of just a continuation block and a goto, we
have a structured terminal. The LabelTerminal expresses the block for the
labeled statement and the continuation, and we can use this to put it back
together when constructing a ReactiveFunction. Note that this PR is just the
scaffolding for LabelTerminal, the next PR is the interesting bits.
Adds a script to automate adding/updating the copyright header to all
appropriate files. For now i've excluded fixture inputs, just because it would
impact fixture outputs too, but we can add them in a later PR if we want.
Type inference currently assumes that a `FunctionSignature`'s effects have no
false positives. If a `mutate` effect is observed on a read-only place, Forget
currently assumes this is an user error and
[throws](207595e04e/forget/src/Inference/InferReferenceEffects.ts (L275-L281)).
Array.from is polymorphic -- its effects are dependent on the type of its
parameters
This PR ensures that we use a single id space for the `BlockId`s in both
top-level functions as well as any nested FunctionExpressions (note, we already
do this for `IdentifierId`). This will make it easier for follow-ups to merge
the CFG of nested functions (ie useMemo bodies) with the parent without block id
collisions.
Fixes `<fbt>`. This required a bunk of yak shaving to work through several
issues:
* First, there was a bug in codegen for JsxNamedspacedName. I added handling for
it for identifiers, but JsxNamespacedName gets converted to a Primitive. The
output looked correct because Babel happily creates invalid Jsx identifiers!
* Next, I needed to add locations to JSX nodes. It took me a while to pinpoint
which specific node needed the location, so I ended up just adding locations to
all the parts of a Jsx element.
* That uncovered the fact that FBT was expecting the `<fbt:param>`'s `name`
attribute value to be a StringLiteral, not a StringLiteral wrapped in a
JsxExpressionContainer. So now we special-case JsxAttribute and emit raw
StringLiteral (either is allowed per the spec)
And with that, voila, `<fbt>` works.
Adds the two FBT (https://facebook.github.io/fbt/) plugins to our test setup so
that we can verify Forget plays well with FBT. Unfortunately FBT's plugins are a
bit finicky, and things that are technically allowed per the JSX spec (such as
wrapping string attribute values in a JsxExpressionContainer) aren't supported
by FBT's plugin. This PR is just to add the fbt plugins and highlight some cases
that fail; these are fixed in later PRs in the stack.
For example, the `fbt-params.js` fixture fails on this PR:
Input
```error.fbt-params.js
import fbt from "fbt";
function Component(props) {
return (
<fbt desc={"Dialog to show to user"}>
Hello <fbt:param name="user name">{props.name}</fbt:param>
</fbt>
);
}
```
Output
```
React Forget › __tests__/fixtures/compiler › fixtures › fbt-params
Expected fixture 'fbt-params' to succeed but it failed with error:
/Users/joesavona/github/react-forget/forget/fbt-params: fbt: unsupported babel
node: MemberExpression
---
props.name
---
```
See fixes later in the stack.
Adds support for `for` statements with an empty or unreachable update
expression. In both cases, reversePostorderBlocks() will remove the
empty/unreachable update block, leaving the ForTerminal.update pointing to a
non-existent block. We explicitly rewrite this (much like we null out
unreachable fallthroughs after shrink). When transforming to ReactiveFunction,
we emit the update block as null if it was the same as the test block.
Fix for the previous issue, suggested by @gsathya: when we run
InferReferenceEffects on the outer function we check each closure to see if it
actually captured any mutable values. If it didn't, we can mark the closure as
readonly and memoize it independently.
Repro of a closure that we currently treat as readonly because it captures a
possibly-mutable value, but which we later realize is not mutable. Specifically,
when we check `exit()` we think `dispatch()` is mutable and therefore consider
it captured, which means we can't independently memoize `exit`.
Updates `PruneNonEscapingScopes` to consider hook arguments as potentially
escaping. This is because hook inputs are "owned" by React — for example,
closures passed to `useEffect`, or a value that is passed to a custom hook and
which then becomes a memoized input.
Similar to what we did for `<fbt>` jsx elements, this PR ensures that `fbt()`
calls have their operands memoized in the same scope to honor the limited
contract for what's allowed as an argument of an fbt() call expression.
There are some internal restrictions in Metro that only allow us to specify one
gating module as an injected dependency. To allow multiple projects, this PR
updates the Babel plugin to take a gating options config specifiying a project
name. The project name is used as a suffix for the generated import; for
example:
```js
const options = {
// ...
gating: {
module: "ReactForgetFeatureFlag",
importSpecifierName: "isForgetEnabled_Secret",
};
// generates
import {isForgetEnabled_Secret} from "ReactForgetFeatureFlag"; // a module that
exports multiple flags
// ...
```
This is a Meta-ism, but adding it for now to unblock. We special-case the
`<fbt>` element for translation purposes, and have a transform that requires the
children of this element to be a limited subset of nodes. Notably, any dynamic
translation values must appear as `<fbt:param>` children — we disallow
identifiers as children of `<fbt>` nodes.
This PR adds a new pass which finds `<fbt>` nodes and ensures their immediate
operands are not independently memoized. Note that this still allows the values
of `<fbt:param>` to be independently memoized, as demonstrated in the unit test.
```js
// here, `a?.b.c` is a single optional chain
// (evaluates to undefined if a is nullish)
a?.b.c;
// here, 'a?.b` is an optional chain, and `.c` is an unconditional load
// (nullthrows if a is nullish)
(a?.b).c;
```
---
Next PR in stack will add a bailout for `(a?.b).c`.
(If we want to properly handle `(a?.b).c`, we might want to model optional
chains explicitly in the HIR. We currently assume that any `PropertyLoad` whose
lhs is an optional property load is read conditionally.)
This is incredibly obvious in hindsight, but for exposure logging to work
correctly we need to *call* the underlying `MobileConfig.getBool` function at
the callsite – otherwise the bool is evaluated once (and only once) when the
module is loaded.
Tested internally and verified that in dogfooding the exposure logging was
working correctly
- Fixes a missing break in InferTypes - I disabled no-fallthrough previously
because it would erroneously report that certain cases with non-builtin throws
(eg `invariant`) would fall through. This brings the rule back but allows
disabling it with a `// break omitted` comment, since it's still helpful in
catching some actual missing breaks.
- Add `DEFAULT_SHAPES` ShapeRegistry, which holds builtins and all
`ObjectShapes` used in `DEFAULT_GLOBALS`.
- Add a few typed objects / functions into `DEFAULT_GLOBALS` (used for tests)
- Add type inference and `infer-global-object` test
Adds `GlobalRegistry`, which holds the names and types of known global objects,
i.e.
```js
type GlobalRegistry = Map<string, PrimitiveType | ObjectType | FunctionType |
HookType | PolyType>;
// ...
globalRegistry.get("NaN"); // {kind: "Primitive"}
globalRegistry.get("parseInt"); // {kind: "Function", shapeId: "..."}
globalRegistry.get("Math"); // {kind: "Object", shapeId: "..."}
```
Since we currently do not track module imports and module-level declarations,
builtin and custom hooks currently also live in GlobalRegistry.
```js
globalRegistry.get("useState"); // {kind: "Hook", definition: {...}}
globalRegistry.get("useFreeze"); // {kind: Hook, definition: {...}}
```
This PR does not allow Forget users to define their own globals. When we add
this as a configuration, we should not expose `ShapeRegistry` to the user, as a
user-provided ShapeRegistry may accidentally be not well formed. (i.e. missing
(1) required shapes (BuiltInArray for [] and BuiltInObject for {}) or (2) some
recursive shapeIds)
```js
export type UserType = UserObject | UserFunction | "Primitive" | "BuiltinObject"
| ...;
export type UserObject = {
kind: "Object",
properties: Map<string, UserType>
}
export type UserFunction = {
kind: "Function",
properties: Map<string, UserType>,
signature: ...
}
export type UserGlobals = Map<string, UserType>;
class Environment {
constructor(globals: Map<string, UserType>, ...) {
// ...
addUserDefinedGlobals(this.#globals, this.#shapes);
```
---
Simplify Environment options by:
- EnvironmentOptions -> EnvironmentConfig
config is now directly passed around instead of being eagerly merged.
- Moving merging / initialization logic into `Environment` constructor. From my
understanding, there is no need to decouple merged options from an environment.
This prepares Environment for the next PR, which adds non-stateful properties to
Environment (i.e. a `GlobalRegistry`) that should be converted from config
values (i.e. not directly exposed to the user due to potentially inconsistent
inputs)
---
#1254 added inference for hooks loaded from globals. This is the only time we
need to generate a type equation assigning `lval` to a resolved`Hook` type.
@gsathya Would love to get your feedback here on the change. From my
understanding, this change is technically incorrect, since the type equation we
generate should be dependent on the `callee` type (i.e. `Hook` if callee is a
hook, `Function` if callee is a function).
Would the next step be to consolidate `Hook` and `Function` types?
```js
type Function {
...
isHook: boolean, // set by inference
}
type FunctionSignature {
isHook: boolean, // set when adding to ShapeRegistry
}
```
I already taught `lowerAssignment()` to handle assignment patterns for
destructuring, we just have to call this helper for assignment pattern params
too.
In a lambda, a return/throw terminal could return a captured context ref needs
to be treated as a mutation to correctly alias the returned context ref and the
lvalue.
Terminal operands are generally not mutating so this hasn't mattered so far. But
in a lambda, a return terminal could return a captured context ref which needs
to be treated as a mutation to correctly alias the returned context ref and the
lvalue.
`JSXEmptyExpression` is never added to a React element's children [in
`react.buildChildren`](https://github.com/babel/babel/blob/main/packages/babel-types/src/builders/react/buildChildren.ts),
which is [used
by](https://github.com/babel/babel/blob/main/packages/babel-plugin-transform-react-jsx/src/create-plugin.ts#L649)
`plugin-transform-react-jsx`].
An alternative would be to represent JSX expressions differently in HIR, then
codegen `JSXEmptyExpression`s back when we encounter an `EmptyExpression`
```js
- children: Array<Place>,
- children: Array<Place | "EmptyExpression">,
```
(We could also retain `JSXEmptyExpression` as an `InstructionValue` that
produces a Primitive. However, this would make babel types in Codegen a bit more
messy, as `JSXEmptyExpression` does not extend `Expression` (which currently is
the result of every `InstructionValue`).)
Creates a new helper, `const temp: Place = lowerValueToTemporary(builder,
value)` which creates a new temporary and an instruction to write that value to
the temporary. We have this pattern all over BuildHIR, and the new helper makes
this all a bit tidier.
Adds support for `await` expressions. We have primarily seen await used inside
callbacks, not directly within component render logic, but because we construct
HIR for lambdas it is helpful to be able to model await rather than require
everyone to rewrite to use the Promise API. Note a subtlety: awaiting a promise
is a mutative operation, so we a) model it as a Mutate effect and b) avoid DCE
of await expressions since they may cause side effects. See the test cases for
examples.
Adds a new helper method that we can use when processing expressions whose
evaluation ordering may not be preserved. This was previously the case only for
switch test case values, but we can use this for AssignmentPattern
(destructuring default values) as well.
"Supports" default values in destructuring (AssignmentPattern) by lowering to a
ternary, even in the output. Examples:
```javascript
// Input:
const [x = 'default'] = y;
// Output:
const [t0] = y;
const x = t0 === undefined ? 'default' : t0;
```
```javascript
// Input 2
const [{x} = makeObject()] = y;
// Output 2
const [t0] = y;
const {x} = t0 === undefined ? makeObject() : t0;
```
Note that this is how Babel lowers AssignmentPattern, so it isn't too bad. This
should help avoid the need to update product code, even if the output isn't
perfectly ideal.
This is kind of a hack, but i think it's worth it given that JSXNamespacedName
is relatively uncommon. Adding a new InstructionValue variant to represent a
namespaced name is one option, but then that isn't a valid expression and can't
appear as an operand anywhere else. Instead, we lower namespaced names as a
primitive (string) as `${namespace}:${name}` — exploiting the fact the namespace
and name can't have a colon, and non-namespaced tagnames also can't have colons.
It's a bit of a hack but it's contained to the JSX processing code. If folks
have strong opinions on this i'm happy to change but this felt reasonable as a
quick and reliable way to unblock support.
NOTE: there is a larger question of what to do about compiling `fbt` tags.
Before we can do anything with them, though, we need to parse them.
We need to check reactivity of both the operand and its resolved source (if
operand is produced by a LoadLocal / PropertyLoad / ComputedLoad).
Both the operand and its source can have reactivity.
e.g.
```js
const o = makeObject(); // source has no reactivity
const x = o[props.x]; // x is reactive
```
Rather than having a special FunctionCall type that deduces the return type,
change the FunctionType to include the return type.
This return type is inferred as part of unification.
---
Every `OptionalMemberExpression` rvalue has the form
`<requiredPath>?.<optionalPath>`.
```
// required = [a], optional: [b, c]
props.a?.b.c;
props.a?.b?.c;
```
When calculating reactive dependencies, recall that it is always correct to add
a subpath of a dependency (e.g. we can always take `props.a` instead of
`props.a.b` as a dependency). See comments in `DeriveMinimalDependencies` for a
longer explanation.
There are two ways we can deal with `OptionalMemberExpression`:
- We can always truncate a OptionalMemberExpression dependency to its
`requiredPath`, taking only the required path as a dependency.
- this is the simpler approach, but it potentially loses granularity.
e.g.
```
// here, since props.a is already unconditionally accessed,
// we can safely add props.a.b as a dependency and preserve both
// nullthrows and the correct dependency set.
scope @0 {
let x = [];
x.push(props.a?.b);
x.push(props.a.b);
}
```
(See added test case `reduce-reactive-cond-memberexpr-join` + its comment block
for a more detailed explanation`
- (the approach taken by this PR)
We can add the `requiredPath` as a potentially unconditional access (dependent
on other control flow) and `requiredPath + optionalPath` as a conditional
dependency.
---
Previously, both `path=null` and `path=[]` could represent a dependency with no
property path (i.e. the result of a LoadLocal with no PropertyLoad).
Make path non-nullable so we don't have to add null checks everywhere.
---
We don't need to store whether a `PropertyLoad` happens within a conditional
(within its reactive scope). In fact, the PropertyLoad producing a rval often is
in a different ReactiveScope from where the rval is used.
We only need to add `#inConditionalWithinScope` when we actually visit a
reactive dependency.
Earlier PRs bailed out when the callee of an OptionalCallExpression was a
MemberExpression or OptionalMemberExpression (ie for optional method calls).
This PRs expands support for optional method calls, including when the receiver,
method, or both are optional. Even better, we don't need to add any additional
terminals or instruction variants for this case - the one new OptionalCall
terminal from earlier in the stack works for all these cases.
Tests, focusing on two key behaviors:
* Dependencies of the args are treated as conditional, since the call may not
happen
* Args cannot be memoized independently, even when that would be valid for a
non-optional call.
Implements HIR->ReactiveFunction conversion and Codegen for optional calls. We
add a new OptionalCall variant of ReactiveValue, which is a SequenceExpression
that describes the evaluation of the args and the call itself. This is then
straightforward to codgen.
Implements lowering for a subset of optional calls - specifically, we don't
(yet) support when the callee is a member expression or an optional member
expression. So `foo?.()` works but we bailout on `object?.foo()` and
`object.foo?.()`.
For `<calleee>?.(<args>)` we lower as roughly:
```
bb0:
t0 = <callee>
OptionalCall test=bb1 fallthrough=
bb1 (value):
Branch t0 consequent=bb2 alternate=bb3
bb2 (value):
...lower <args> here...
t1 = Call t0, args
StoreLocal res, t1
Goto bb4
bb3 (value):
t2 = undefined
StoreLocal res, t2
Goto bb4
bb4:
// result in `res` here
```
Adds a new `optional-call` terminal and sets up the appropriate handling in the
visitors, with lowering/reactivefunction/codegen as todos for now and
implemented in follow-ups.
---
Expand Hindley Milner type inference to infer dependent types.
Say `t` is a typevar and `t'` is some type (a built-in type, phi node, or
another typevar).
Our type equations are as follows (please edit/correct notation 😅)
- type substitution: `t = t'`,
- ~~dependent~~ polymorphic property load: `t = t'.prop`
- polymorphic function call `t = fnCall{returnType}`
- ~~dependent property call: `t = t'.prop` (only if t'.prop is a function
type)~~
- ~~dependent return type: `t = t'.[[returntype]]`~~
---
+10 −1,698 lines [[insert impacc macro]]
The ObjectShape stacks (#1350, #1358) used these tests to record changes in
inferred types (and associated ObjectShapes), reference effects, and mutable
ranges.
Now that those PRs have landed, we can delete these tests. They are somewhat
fragile (changing anytime HIR / printHIR is changed) and easily cause
rebase/merge conflicts.
---
This PR does not add inference for normal `CallExpression`s, since built-in
functions for `Array` and `Object` are usually only valid if called with a
correctly-typed `this`. If we want codegen to preserve source code semantics,
Forget should only add inferred types it is confident about.
This PR also adds `returnEffect` to FunctionSignature. `returnEffect = Store` if
this function is known to always return a captured value from `receiver` or
`args`.
---
Expand Hindley Milner type inference to infer dependent types.
Say `t` is a typevar and `t'` is some type (a built-in type, phi node, or
another typevar).
Our type equations are as follows (please edit/correct notation 😅)
- type substitution: `t = t'`,
- ~~dependent~~ polymorphic property load: `t = t'.prop`
- polymorphic function call `t = fnCall{returnType}`
- ~~dependent property call: `t = t'.prop` (only if t'.prop is a function
type)~~
- ~~dependent return type: `t = t'.[[returntype]]`~~
We limit the types of expressions allowed as switch case test values because we
our HIR doesn't yet preserve order-of-evaluation for switch test values (we
model them as being evaluated prior to entering the switch, as opposed to
lazily, when the case is reached). One common pattern internally is test case
values that are properties of a global, eg you have some bag of enum values and
are comparing against that:
```javascript
// at module scope, or imported from another module:
const OPTIONS = {FOO: 'foo'};
// in a component
switch (value) {
case OPTIONS.FOO: { ... }
}
```
This PR allows this specific case, ie member expressions where the innermost
object is a global identifier.
Now that we model the method resolution via a PropertyLoad or ComputedLoad, we
don't need to distinguish between PropertyCall and ComputedCall. These two call
variants are now combined into a single MethodCall variant.
This is the version of @mofeiZ's change for PropertyLoad, but made to work on
ComputedCall. We force the method to be evaluated in the same scope as the call
in InferReactiveScopeVariables.
---
(I'm not sure if these are already known issues. I found them while playing
around with lambda captures. They are also reproducible on main / stable)
I have some limited understanding of lambda captures after reading Sathya's
posts -- please correct if/where this is incorrect
```
function Component() {
// instr1
// instr2
const func3 = function(...) {
// func3instr1
}
}
```
We currently determine effects of captured references in `AnalyzeFunctions`,
before InferReferenceEffects.
- i.e. for some function
1. dependencies of all functions (func3.deps)
2. prefix traversal of all instructions (e.g. instr1, instr2, func3.deps,
func3instr1, ...)
- is this just an implementation decision? i.e. what is stopping us from postfix
traversal in InferReferenceEffects (e.g. instr1, instr2, func3instr1,
func3.deps)
As such, for each captured reference, `AnalyzeFunctions` needs to assign a
reference effect. We currently check `MutableRange`, which seems to miss a few
cases
- We do not model assignments to primitives correctly, since primitives do not
have a mutable range.
- We're not able to model captured (but not mutated) values correctly.
Would it be possible to consolidate `AnalyzeFunctions` into
InferReferenceEffects, using some post-order traversal (iterating over a
function's instructions to collect its dependencies + associated capture
effects)? I definitely don't understand lambdas completely, so please tell me
what I'm missing
---
This PR does not add inference for normal `CallExpression`s, since built-in
functions for `Array` and `Object` are usually only valid if called with a
correctly-typed `this`. If we want codegen to preserve source code semantics,
Forget should only add inferred types it is confident about.
This PR also adds `returnEffect` to FunctionSignature. `returnEffect = Store` if
this function is known to always return a captured value from `receiver` or
`args`.
---
I didn't properly understand Capture and Store effects previously, just
correcting those mistakes!
These functions are all synchronously mutative, so they should use Read /
Mutate, not Capture + Store
---
Expand Hindley Milner type inference to infer dependent types.
Say `t` is a typevar and `t'` is some type (a built-in type, phi node, or
another typevar).
Our type equations are as follows (please edit/correct notation 😅)
- type substitution: `t = t'`,
- ~~dependent~~ polymorphic property load: `t = t'.prop`
- polymorphic function call `t = fnCall{returnType}`
- ~~dependent property call: `t = t'.prop` (only if t'.prop is a function
type)~~
- ~~dependent return type: `t = t'.[[returntype]]`~~
How Forget currently lowers PropertyCall:
```js
// source: [[ calleeExpr ]].propertyName( [[ argExpr0 ]])
$0 = [[ calleeExpr ]]
$1 = [[ argExpr0 ]]
$2 = PropertyCall callee=$0 property="propertyName" args=[$1]
```
This PR changes the lowering:
```js
// source: [[ calleeExpr ]].propertyName( [[ argExpr0 ]])
$0 = [[ calleeExpr ]]
$1 = PropertyLoad $0 "propertyName"
$2 = [[ argExpr0 ]]
$3 = PropertyCall callee=$0 fn=$1 args=[$2]
```
From my understanding, `PropertyCall` needs the receiver to properly model JS
semantics which is something like `resolvedFn.apply(resolvedCallee, arg0, arg1,
...)`. This is additionally useful for:
- Fine-grained mutability / alias analysis. The property call is technically a
read of the resolved function, and a mutate of the callee.
- Dependency tracking. While we could special case PropertyCall, this
representation would correctly add both callee and callee.propertyName as
dependencies for PropertyCall.
e.g.
```js
let x = [];
mutate(x);
useFreeze(x);
let y = {};
y.a = x.bar();
return y;
```
Reverts #1199, which was added before we properly supported destructuring
assignment.
Next PR (changes to PropertyCall in #1384) will lower two references to the same
named identifier (the property call receiver)
The previous PR only updated simple assignment expressions (where the lvalue is
an identifier), this PR extends the same idea to all assignment variants. Note
that there is one case that doesn't work yet, which is complex destructuring
assignment as a value:
```javascript
let x = makeObject();
x.foo(([[x]] = makeObject()));
```
What happens here is that we lower the destructuring to a series of steps:
```
tmp1: Destructure Const [ tmp0 ] = makeObject();
tmp2: Destructure Reassign [ x ] = tmp0;
PropertyCall x, 'foo', [ tmp1 ]
```
Thankfully we can detect this case: if we have a const/let declaration with an
lvalue, that's invalid. See the new error test case which shows we correctly
detect & reject this case for now.
This PR subtly changes how we represent assignment expressions in order to
accurately model them _as expressions_. Specifically, the result of lowering an
assignment is now the temporary created for the assignment's lvalue. This allows
us to restore the assignment as a value (expression) during codegen. Note how
this fixes a bug and cleans up some output.
Updates ConstantPropagation so that each instruction is responsible for whether
to replace its `.value` with the resolved constant value (if found).
Specifically, for `StoreLocal` we don't want to replace the value — we want to
keep the assignment — but we do want to propagate the _result_ of the assignment
downstream. This more accurately models the semantics of assignment expressions,
and helps with subsequent PRs.
Rewrite ArrowFunctionExpression to FunctionDeclaration and compile it. This lets
us reuse all the export gating logic, rather than writing separate, specific
logic for ArrowFunctionExpression.
Cleans up duplicated code for processing call/constructor arguments. As a side
benefit, we now support spread elements for constructor arguments (and if we
want to change how we represent that, we can do it in one place).
We currently are not lowering property calls in evaluation order.
I wonder if the following lowering for a PropertyCall (with static or computed
property) is semantically equivalent to source:
1. eval + resolve receiver (store in t0)
2. eval computed property (if present)
3. resolve t0.property (binding the call to receiver and storing in t1)
4. eval args
5. eval t1(args)
Although codegen might then generate something like this (if args are named
temporaries)
```js
const tmp = receiver.property.bind(receiver);
// lower args to temporaries
tmp(arg1, arg2);
```
Adds a `DeclareLocal` instruction which represents declaring a named variable
without initializing it. Currently declarations without an initializer (`let x`)
are transformed into a declaration to undefined (`let x = undefined`) which
changes the semantics due to hoisting and TDZ (temporary dead zone). The correct
thing is to represent declaration without initialization.
These examples previously errored all the way in codegen, when we detected that
a value block (eg a `while` test expression) was declaring a new variable. We
now detect this in LeaveSSA and error. The actual fix is a bit tricky, we'd need
to add a new declaration in the nearest block scope (or selectively not DCE the
declaration if its reassigned in just this way).
If a logical or conditional expression is unused, then a phi node isn't created
for the identifier it assigns to. Then when we leave SSA form the two branches
will assign to separate values, and we aren't sure which identifier to use as
the lvalue of the resulting ReactiveInstruction (remember that
logicals/conditionals decompose into control flow in HIR, but are a single
compound instruction in ReactiveFunction). If the two sides don't assign to the
same location, it could be because of a bug in the compiler or because the value
wasn't used. Ideally we'd represent this explicitly, but for now i'm just making
this a TODO since most logicals/conditionals should have their value used.
Enables support for assignment expressions in value blocks (which includes in
loop init/test/update blocks). This was pretty straightforward, the main changes
are:
* During PropagateScopeDependencies, we currently record scope reassignments
based on `Identifier` object identity. In the case where a variable is
reassigned in multiple control-flow paths of a value block, however, there can
be multiple object identities. So we now de-dupe reassignments based on
identifier id.
* MergeOverlappingScopes now treats value blocks as regular blocks, allowing it
to correctly merge scopes from the value with other scopes from the outer block.
Otherwise this is mostly just lots of tests. Note that there is an outstanding
todo, which is that we currently error for ternaries and logicals whose value is
unused (eg `cond ? (x = 1) : null`). I'll address that in a follow-up.
This PR clarifies the logic for adjust mutable ranges of phis and their operands
during LeaveSSA. Previously we had logic in several places to determine
whether/how to extend the ranges of each phi and its operands: this occurred
while traversing reassignmentPhis (in 2+ places) and rewritePhis, as well as in
rewritePlace().
This was kind of a band-aid to make things work, but the logic was imprecise.
The actual rules are as follows:
If there is a back-edge, or the phi id is unnamed, then were extend the ranges
of the phi and its operands to min(starts) and max(ends). This ensures that the
operands are computed as one unit, ie put into a single reactive scope. For
loops this is necessary because...looping! For unnamed values this is necessary
because of the way we collapse logical and ternary expressions back to a
hierarchical ReactiveFunction — we need to make sure the final mutable range
extends from the start of the final instruction up to the end of the
logical/ternaries value blocks.
Otherwise this is a phi where operands come from predecessors and are named. If
the phi is mutated later, then we have to extend the end of each operand's range
to account for the fact that they can be mutated later. Else, we leave the
operands alone.
Behavior doesn't change, but we consolidate all of the mutable range logic in
one place.
Makes JSX memoized by default again, but adds an option to disable memoization
of JSX. Also adds a new test and fixtures directory to test the opt-in
no-jsx-memoization behavior.
---
Currently, we run type inference passes early in the pipeline and do not check
inference output in any tests, test fixtures, or verifier passes. In fact, the
only ways to view inferred types are (1) locally add a test fixture with`@only`
and inspect console logs or (2) scroll to the relevant section on a playground
example.
However, inferred types and effects significantly affect the output of later
passes (Alias / MutableRange analysis, InferReactiveIdentifiers, etc), and we
have already found some bugs due to incorrect inference (e.g. #1274).
This PR add the `typer-tests` fixture with the following goals
1. Record relevant current compiler type + effect inference output.
2. Have relatively stable output (with respect to changes in HIR and PrintHIR).
- we try to achieve this by annotating the source code.
---
Currently, we run type inference passes early in the pipeline and do not check
inference output in any tests, test fixtures, or verifier passes. In fact, the
only ways to view inferred types are (1) locally add a test fixture with`@only`
and inspect console logs or (2) scroll to the relevant section on a playground
example.
However, inferred types and effects significantly affect the output of later
passes (Alias / MutableRange analysis, InferReactiveIdentifiers, etc), and we
have already found some bugs due to incorrect inference (e.g. #1274).
This PR add the `typer-tests` fixture with the following goals
1. Record relevant current compiler type + effect inference output.
2. Have relatively stable output (with respect to changes in HIR and PrintHIR).
- we try to achieve this by annotating the source code.
We were treating Destructuring as if it could never allocate and therefore
didn't have to be memoized. That's only true if there are no rest spreads
though. This PR teaches the compiler to treat rest spreads differently for
scoping and memoization
purposes, fixing the newly added test case and some existing bugs.
Adds a new pass that uses escape analysis and React-specific heuristics to tune
the amount of memoization applied. Specifically, the pass ensures that we only
memoize:
* Values which escape (are directly returned or transitively aliased by a
returned value)
* ...and that are not JSX elements
* OR values which are _dependencies_ of scopes that produce an escaping value.
The latter case is necessary to avoid breaking memoization of an escaping value
bc a scope happened to have a non-escaping dependency.
## Algorithm
1. First we build up a graph, a mapping of IdentifierId to a node describing all
the scopes and inputs involved in creating that identifier. Individual nodes are
marked as definitely aliased, conditionally aliased, or unaliased:
a. Arrays, objects, function calls all produce a new value and are always marked
as aliased
b. Conditional and logical expressions (and a few others) are conditinally
aliased, depending on whether their result value is aliased.
c. JSX is always unaliased (though its props children may be)
2. The same pass which builds the graph also stores the set of returned
identifiers
3. We traverse the graph starting from the returned identifiers and mark
reachable dependencies as escaping, based on the combination of the parent
node's type and its children (eg a conditional node with an aliased dep promotes
to aliased).
4. Finally we prune scopes whose outputs weren't marked.
This was the actual bug. When EliminateRedundantPhis eliminates a phi, it has to
rewrite downstream usages of the phi id to the single operand id. We were
correctly doing that in all but one place. When we iterate _downstream phis_, we
were looking up the operands against the rewrite table, but not updating the phi
operands themselves to the rewritten value.
This fixes the bug, and incidentally fixes a test that has been broken for a
while and nagging at me.
Found while debugging the previous issue: `mapInstructionOperands()` should not
look at lvalues. The previous version was causing us to create extra phi nodes,
which interestingly weren't the actual problem behind the "SSA" bug, but sure
looked like it at first.
Refactors the representation of ObjectExpression properties from a Map to an
`Array<ObjectProperty>` to prepare for the next diff which adds spread element
support.
BuildHIR currently propagates UnsupportedNodes for collection types where the
element itself can fail (for example object expressions where the key may not be
valid). However, given that we currently abort compilation after the first
failing pass (and will probably do so for quite a while) I think we can simplify
and just always return the collection. Note that I already did this for
destructuring. I'm open to leaving the code as-is if you prefer, though.
This PR starts to clean up our handling of lvalues and rvalues by adding new
`eachInstructionLValues()` and `mapInstructionLValues()` helpers. Now,
`eachInstructionOperand()` and `mapInstructionOperands()` only visit true
rvalues, and the new passes must be used to visit lvalues. This allows us to
remove the special-casing for StoreLocal and Destructure in most of the passes.
Instead of replacing original function with compiled code, this adds an option
to append the code and switch between the two based on an `isForgetEnabled` test
condition that's imported from the specified gatingModule.
---
**This PR slightly changes the semantics of ReactiveScopeDependencies**.
Previously, reading a ReactiveScopeDependency is guaranteed to preserve the
`nullthrows` semantics of its own declarations (not that of its inner scopes).
This does not affect the overall correctness properties, since we already hoist
reading of conditional dependencies (and thus may throw earlier than the
original source).
E.g. we already do not preserve *where* the nullthrows occurs.
```javascript
function Component(props) {
// throws here, before print(x)
const c_0 = props.a.b !== $[0];
let x;
if (c_0) {
x = {};
print(x);
if (...) mutate1(x, props.a.b);
mutate2(x, props.a.b);
// ...
```
### Summary
This is an optimization, not a correctness property.
When propagating reactive dependencies of an inner scope up to its parent, we
want to *retain information about conditional dependencies* -- not the derived
unconditional dependencies. This helps us produce more granular dependencies in
the parent scope.
Current implementation:
```javascript
const innerScopeDeps = innerScope.depTree.deriveMinimalUnconditionalDeps();
for (const dep of innerScopeDeps) {
currentScope.depTree.addDep(dep);
}
```
New implementation:
```javascript
// union of a tree takes union of each node
currentScope.depTree = currentScope.depTree.union(innerScope.depTree);
```
### Example
In the below example:
- `scope @1` has a conditional dependency of `props.a.b`, but that reduces to
the unconditional dependency `props`
- `scope @0` itself has a unconditional dependency of `props.a.b`
- Currently, Forget joins the derived / reduced dependencies of inner scopes,
which adds `props` as unconditional dependency of `scope @0`
- With this change, Forget joins the property trees and retains info about
conditional deps, which adds `props.a.b` as a conditional dep of `scope @0`.
```javascript
// scope @0 (deps=[???] decls=[x, y])
let y = {};
// scope @1 (deps=[props] decls=[x])
let x = {};
if (foo) mutate1(x, props.a.b);
mutate2(y, props.a.b);
```
### Followup
We currently keep track of properties unconditionally accessed per
ReactiveBlock. Eventually we want to keep track of properties unconditionally
accessed across blocks (as according to control flow).
Consider the following code, in which sibling scopes 0 and 1 are sequentially
executed. In this case, we can safely add props.a.b as a dependency of scope 1.
```javascript
// scope@0 (deps=[props.a.b], decls=[x])
let x = { a: foo(props.a.b) };
// scope@1 (deps=[???], decls=[y])
let y = {};
if (...) {
mutate(y, props.a.b);
}
```
---
Implementation details summarized in comments.
Overall, we want to calculate a `ReactiveDependencyTree` for every conditional
block. If we know that conditional blocks are exhaustive (e.g. all CFG paths
calculates a tree), we can take `intersection(depsFromEachBlock)` and add this
to the parent Reactive + conditional scope `parentDeps = union(parentDeps,
intersection(...))`.
We use trees instead of individual deps here because we can still derive
unconditional accesses.
e.g.
```
let x = {};
// props.a is an unconditional access here
if (foo(other)) {
x.a = props.a.b;
} else {
x.b = props.a.c;
}
```
---
Small refactor of reactive dependency logic, no behavioral change.
- Moves ReactiveDependencyTree logic into `DeriveMinimalDependencies`.
- this moves hides most helper functions + types 🥳
- made `ReactiveDependencyTree` a class
- Changes `#dependencies` type: `Set<ReactiveScopeDep>` ->
`ReactiveDependencyTree`
- instead of collecting all dependencies into a tree in the end, we now eagerly
join dependencies into the tree on `visitDep`
- this is needed for the next PR in the stack, which relies on incremental
merging
In #1287 i implemented basic handling for destructuring in DCE: if any of the
pattern values are used, we retained the whole instruction as-is. However,
ideally we could prune out unused elements from the pattern. There are pretty
simple rules:
* ArrayPattern we can eliminate unused elements from the end.
* ObjectPattern we can eliminate any unused element, but only if there is no
rest element.
This is more followup toward deleting Instruction.lvalue. The previous PR
ensured that all Instruction.lvalue identifiers are only ever assigned to once.
Now we ensure that `lowerExpression()` is _only_ called via
`lowerExpressionToTemporary()`, ie we now always lower every single expression
to a temporary.
This will make it easier to make lvalue a part of the InstructionValue instead
of the instruction itself, in follow-up PRs.
As part of removing Instruction.lvalue we need to ensure that it is only used to
represent that instruction's value — the InstructionKind should always be Const.
The one place where we violated this was for value blocks, specifically
ConditionalExpression and LogicalExpression. For both of those, we generate a
single temporary place to represent the expression result. Then the consequent
and alternate branch ended in a `LoadLocal` that reassigned that temporary (in
the lvalue) to the result of that branch.
This PR changes to use StoreLocal instead, and updates the recently added
validation pass to ensure that all identifiers that appear in an
Instruction.lvalue are only ever assigned once.
Changes to explicitly model destructuring (array and object patterns), expanding
support to include rest elements and preserving destructuring through the
output. The new "Destructure" instruction is similar to "StoreLocal" but has a
pattern instead of a place. For now each level of nested array/object patterns
creates a separate destructure instruction, which ensures we have a temporary
Place to talk about the intermediate array/object and its type/effects etc.
Example:
```
// INPUT
const [x, {y}, ...z] = a; // yay rest elements work now!
// HIR
[1] <unknown> $2 = LoadLocal a$1
[2] <unknown> $6 = Destructure Const [ <unknown> x$3, <unknown> $4, ...<unknown>
z$5 ] = <unknown> $2
[3] <unknown> $8 = Destructure Const { y: <unknown> y$8 } = <unknown> $4
// OUTPUT
const [x, t0, ...z] = a;
const {y} = t0;
```
Note that we can still collapse to a single destructure statement during
codegen, independently of whether we have separate instructions internally. For
now i'm going w the simple approach of emitting multiple statements in codegen
(the code will very likely get further rewritten by downstream babel passes
anyway).
Also, I don't love the "if StoreLocal/Destructure else ..." pattern that the
StoreLocal created and that this PR entrenches. As discussed w @gsathya offline,
the long-term direction will be to add a separate visitor, roughly
`eachLValue()` and `eachOperand()` so that we can treat all instructions the
same. Existing Instruction.lvalue will go away and become a property of the
other types of instructions.
InferMutableRangesForAlias is about extending the mutable ranges, not for
updating the alias sets. Let's refactor this into a separate pass.
InferMutableRangesForAlias was iterating over alias sets and not the HIR so this
refactor isn't costing us any additional perf cost (in terms of an extra
iteration over the HIR).
Identifiers don't need skipping anyways, so this doesn't affect the existing
behavior.
In the future, we will special case handling of LHS of AssignmentExpression
which will require us to not skip the RHS.
The mutable range difference for assignment is just 1 which is something we
usually don't track as we care about mutation and not assignment.
But this isn't true for context refs whose (re) assignment is actually a
mutation.
Having undefined as the initial value makes this a primitive. There's a separate
bug where we need to remove type inference for captured refs that get mutated
but that's secondary -- we're currently not even marking the Identifier LHS as a
captured ref. Fix the test to repro this bug for now.
Add a `logger` option so compiler errors are surfaced in our metrics collection
pipeline.
We can probably later merge the global `log` function with it.
Adds a new `StoreLocal <kind> <place> = <value>` instruction which stores
<value> into <place>. With this change, Instruction.lvalue is _always_ a `const`
temporary, and never a named identifier (there's a new validation pass to assert
this). StoreLocal is the only way to declare or update a named identifier: the
instructionKind property says whether it's a const/let declaration or a
reassignment. Naturally a _lot_ of passes had to be updated to make this work,
but the existing Effect.Store variant that @gsathya added made this overall
straightforward.
Note that as of this PR several passes still have code to handle the possibility
of an instruction lvalue being something other than a temporary. When we clean
that up in a follow-up, there will be a lot less of the duplication that appears
here. For example, CodegenReactiveFunction has two places to handle variable
declarations in this PR. However, one of them is to handle lvalues, which should
now _always_ be temporaries and never emit a regular variable declaration.
Similarly, several passes have to build up a table of identifier -> identifier
(because of LoadLocal). Longer-term, we should update the Place abstraction so
that it directly specifies the instruction which created that temporary, so we
can look it up on demand instead of needing an extra mapping.
Adds support for DoWhileStatements. It's pretty similar to how we handle While,
except in the case where a test block is unreachable (for example, an early
unconditional `break` within the loop body). In this scenario we eliminate the
terminal altogether and replace it with a goto to the loop block.
Changes InstructionValue::Place to InstructionValue::LoadLocal for clarity, this
is intended as the only instruction where a variable can appear as an operand.
All other instructions operands will be temporaries.
---
> If this operand is used in a scope, has a dynamic value, and was defined
before this scope, then its a dependency of the scope.
> (from current comments in PropagateScopeDependencies::visitDependency)
A reactive scope can take a dependency from a definition produced by an
incomplete parent scope. Our tests previously did not cover this, since most
object types aliased together and remained mutable throughout a ReactiveScope.
e.g. our tests did not have
```
scope @0 (deps=..., declarations=[x, y]) {
x = {};
// define a reactive, immutable value that is not aliased to become mutable
const immutableVal = ...;
scope @1 (deps=immutableVal, declarations=[y]) {
y = read(immutableVal)
}
mutateX(x, ...);
}
```
We should not add a dependency if it is produced in exactly the same scope as
the one it is used. It is safe (and correct) to depend on values produced by a
parent scope.
---
Note that we still should check for whether a defining scope is active to
determine whether it should be added as a output of that scope
([src](b608ab20d5/forget/src/ReactiveScopes/PropagateScopeDependencies.ts (L469-L478))).
Access of an identifier produced by a parent scope (i.e. adding a variable
defined by a scope's parent as its own dependency) does not require adding that
identifier to the parent's `declarations`, since that identifier is already
valid to access via identifier binding rules.
---
Following #1216:
If a value is known to be immutable, then it doesn't need to be considered
'captured' since no mutation should occur.
Couldn't figure out a unit test in which this specific fix matters, but we need
this to fix test output of #1273
cc. @gsathya, would love some feedback / eyes on this. This makes sense for
Primitives in particular (which are always read / copied in rval position), but
I'm not as familiar with edge cases for other immutable values especially around
lambdas.
---
Our current compiler has specific logic for determining what can be a reactive
value / reactive dependency.
Currently, all of the following affect whether an identifier is a reactive:
- **alias analysis** (applicable to objects)
- **data + control flow** (whether any other reactive identifiers is used in
determining it)
- **reactive scopes** (we generalize and say anything produced by a block with
reactive dependencies must be non-stable and reactive)
- this is not true in the case of const primitives, but an overestimate is safe
- whether the **scope that declares this identifier** is ~~currently active~~
the same scope in which it is used (fixed by #1275)
(since a scope cannot be dependent on itself)
These conditions are complex. We end up inferring most identifiers as `mutable`
and `object` types, which have different stability and aliasing properties from
primitives. As a result, we're missing some cases in our existing test coverage.
Test case output is fixed by #1274 and #1275
---
(This can be separated from the stack below, which implements conditional
dependencies. Happy to merge that first and open this as a new stack if that
produces a significantly better Git PR history.)
---
See comment block in `PropagateScopeDependencies` and added test case
`reduce-reactive-conditional-dependencies` for correctness properties /
dependency merging logic.
---
See comment block in `PropagateScopeDependencies` and added test case
`reduce-reactive-unconditional-dependencies` for correctness properties /
dependency merging logic.
---
We never use the `Place` of a ReactiveScopeDependency, except for when we want
to access its identifier. Later PRs in this stack will convert
`ReactiveScopeDependency` to property access trees (and traverse over the tree).
This usually involves merging multiple Dependencies into trees (where each root
is a unique identifier). We then traverse over each tree to extract its
dependencies (e.g. unconditional leaves).
```
{place: {loc: 1, identifier: 'props'}, path: ['a', 'b']}
{place: {loc: 2, identifier: 'props'}, path: ['a']}
// merges into a single tree root, which should represent a single identifier
```
The `place` of each individual `ReactiveScopeDependency` will be lost during the
tree traversal, and it doesn't really make sense to recreate them using the
`Place` attached to the tree root.
---
Patch and simplify logic around merging overlapping reactive dependencies.
Added `reduce-reactive-unconditional-deps` test fixtures, which tries to cover
all cases of merging unconditional dependencies (to a minimal dependencies set).
Please let me know if I missed any
This PR changes BuildHIR to lower all operands to temporaries. Example:
```javascript
// Input
a + b;
// Previous Lowering
Const t0 = BinaryOperation Place(a) "+" Place(b)
// New Lowering
Const t0 = Place(a);
Const t1 = Place(b);
BinaryOperation Place(t0) "+" Place(t1)
```
This is necessary to ensure we're always referring to the correct version of a
variable, even in the case of reassignment mid-expression. For example, we
previously evaluated `let x=1; x + (x = 2) + x` incorrectly to 6 because we
lowered the `x = 2` prior to the binary operators. We now lowers each instance
of x to a temporary, ensuring they refer to the correct SSA version of the
variable, and produce the correct result (5).
Note that with this change, the _only_ place a variable can appear as an
operator is when the InstructionValue is a raw identifier. This was already the
case for globals (as of the LoadGlobal instruction). All other instruction value
variants will only ever receive temporaries as arguments.
This necessitated a few changes to our inference:
* The logic to extend the range of phi operands (if the phi is mutated) was
previously in LeaveSSA, but that was actually too late. The introduction of
lowering to temporaries help discover failing cases, which I fixed earlier in
the stack by moving the logic to extend the range of phi operands into the
InferMutableRanges fixpoint loop.
* PropagateScopeDependencies now has to track variable reassignments in addition
to tracking property accesses
* AnalyzeFunctions now has to track variable reassignments in addition to
tracking property accesses
* InferReactiveIdentifiers now needs a fixpoint iteration, because identifiers
don't directly appear together in the same instruction anymore (such that we can
directly propagate the reactivity between them). Instead, we'll first see that
the temporaries are reactive, and have to propagate that back to the identifiers
the temporaries were loaded from.
Overall while this does introduce a bit more complexity, it also makes the
compiler more robust. As with the phi example illustrates, there are legitimate
inputs that can create similar indirections to that introduced by lowering
identifiers to temporaries.
Note that there’s a theme to the changes here: several analysis passes need to
map an operand back to its identifier value. Ideally our HIR structure would
directly support looking up the value for a temporary. For example, if operands
were references to eg the index of the instruction that produced them. Because
we don’t have such a representation yet (it would fall out naturally if we were
writing in Rust), we have to do some bookkeeping. The key takeaway here is that
this bookkeeping is incidental complexity given our current representation, not
fundamental complexity of the algorithm.
I found this while working to ensure that we always lower all operands to
temporaries. This works:
```javascript
// the whole computation of x is memoized in one block, bc of the mutation after
the phi
let x;
if (cond) {
x = someObj();
} else {
x = someObj();
}
mutate(x);
```
However, if you alias either of the operands, we lose the mutation:
```javascript
let x;
if (cond) {
const y = someObj(); // OOPS this gets independently memoized
x = y;
} else {
x = someObj();
}
mutate(x);
```
The core issue is that InferMutableRanges does not take into account mutation of
phis. ~~My first thought is that we need an additional, outer fixpoint iteration
loop to flow mutation back "up" to phi operands~~
edit: there was a much easier fix, we need to alias phi operands and phi id
within the existing fixpoint iteration. See follow-up PR which fixes.
This is a precursor to validating that all identifiers are defined - we need to
know about gobals and module declarations, so this PR adds the ability to
configure a Set<string> of defined globals. The default list is inspired by the
globals that prepack defines, which just comes from the spec definition.
Updates BuildHIR to produce LoadGlobal instructions for references to globals.
Note that this breaks our previous strategy of finding hook calls: that relied
on looking at the callee of a CallExpression and checking its name, which relied
on the callee not being lowered to a temporary. By lowering the name (eg
`useState`) to a temporary first, we now no longer see the name at the callsite.
Thankfully @gsathya solved this for us already by teaching type inference about
hooks, and more generally implementing type inference. I updated this so that we
infer the type of a LoadGlobal if the name is a hook: the type inference picks
this up and propagates the type forward correctly. So now, all places that
needed to check for a hook can just look at the type and everything works.
This is much more robust than before - you can now reassign a hook to a local
variable and we'll still detect that when you call it, you're calling a hook.
Adds a new `LoadGlobal` InstructionValue variant which will be used to represent
identifiers that refer to globals. We don't construct this value type yet.
Updates the babel plugin so that environment options — including custom hook
definitions — can be passed in through the plugin:
* Renames `CompilerFlags` => `PluginOptions` since they are specific to the
babel plugin, and are no longer just flags.
* Moves the definition of `useFreeze()` out of the builtin hook list and instead
passes it when our unit tests configure the plugin.
InferReferenceEffects needs to be able to pass around the function's
Environment, but there is already a local class with that name. It's confusing
to have two "environment" concepts in one file, so this PR renames that local
class to the more appropriate `InferenceState` and renames local variables and
updates comments accordingly.
While reviewing @poteto's PR I noticed that there were some cases of missing
dependencies. I tracked it down to a bug I introduced
[here](5b827eb85c (r100646304)).
Decl.id is meant to be the id of the instruction that declares the variable. We
then test to see if a dependency is later than that. If the Decl.id is
incorrectly too high, then we miss some dependencies thinking they aren't
defined yet.
This is to help prep for @poteto's renaming PR. To make that PR work we
generally need to use IdentifierId to distinguish "the same identifier" rather
than Identifier object identity.
InferReactiveIdentifiers has some extra logic to find identifiers declared in
the same scope, and promote non-reactive identifiers to reactive if they appear
inside a reactive scope (reactive scope == scope with one or more (reactive)
dependencies). Even though the identifier alone might not be technically
reactive (have no reactive inputs), it can get re-recreated if the scope
re-evaluates.
We can now do this during PruneNonReactiveDependencies as we exit out of each
scope.
I removed fixpoint iteration and all tests pass, which matches my intuition that
it's really that we need strictly two passes. Removing to simplify and for
performance (avoid unnecessary extra visits of the ast)
The fact that InferReactiveIdentifiers is integrated directly into
PropagateScopeDependencies has made the latter pretty tricky to debug at times.
If a dependency is missing, we have to introspect and figure out if that's
because it was somehow inferred as non-reactive. This PR creates a new
PruneNonReactiveDependencies pass to separate out these phases.
Optimizes dead code elimination. Currently it keeps iterating the control flow
graph until no new usages have been discovered, which accounts for usages across
loops. However, when there are no loops it's sufficient to iterate the CFG
exactly once.
With the upcoming changes to SSA renaming in #1194, we rewrite phi operand
identifiers to have the same IdentifierId as the declaration the identifier
originated from: so downstream checks need to compare ids instead of the
identifier instance.
This tracks whether a value is a context ref or generated from a context ref.
This lets us track mutations to context refs and treat it separately as we want
this to be more conservative than our existing inference.
ValueKind.Context is exactly like ValueKind.Mutable but is more conservative.
This is a temporary fix for the issue we discovered on our first integration,
where destructuring of a function return value is emitting the function call
multiple times:
```javascript
// Input
const [x, setX] = useState(null);
// Output
const x = useState(null)[0];
const setX = useState(null)[1];
```
The reason this happens is that we lower `useState(null)` to a temporary, and
then generate a ComputedLoad for each of x and setX. Codegen doesn't emit
temporaries eagerly - it assumes they are going to be used exactly once and it
re-emits the value each time the temporary is used. Hence why the
`useState(null)` part gets duplicated in the output.
Right now destructuring is the only place i'm aware of where we reuse
temporaries this way. And we do want to change codegen to preserve destructuring
in the output to correctly handle array patterns. However, that's a more
involved change. For now, this PR is a stopgap. During the pass where we promote
temporaries used in scopes to named variables, we now check to see if those
temporaries are used multiple times and promote them.
The above example would then generate something like
```javascript
const t0 = useState(null);
const x = t0[0];
const setX = t0[1];
```
This is still incorrect (it assumes t0 is an array), but it's more likely to
work in practice. I'll revert this change once we correctly handle
destructuring.
Configures typescript-eslint for the project with an initial configuration that
starts with their recommended rules, and adds/disables a few (generally either
disabling warnings or promoting them to errors). The new `yarn lint` command is
not hooked up to CI yet, so for now this is something we can opt-in to running
locally. If you have some free time, help get us down to zero errors!
My general philosophy for linting, which I propose we follow, is that lints
should be very high-signal:
* Error, don't warn. If it's worth mentioning it's worth fixing.
* Enable rules that consistently identify real problems. If we frequently would
have to disable the rule due to false positives, it isn't high-signal.
* Enable rules that help improve consistent style (to avoid code review about
style rather than substance).
I realized we hadn't updated InferReferenceEffects to match our latest thinking
on hooks. Specifically, we will default to assuming that hooks can mutate their
arguments and return mutable values — this works with our model since we don't
treat hooks specially for reactive scope construction. Ie, first we figure out
what variables construct together, then we create scopes, then we prune scopes
that contain hooks. So changing the reference effects for hooks "just works".
Note that it is helpful for our unit tests to have an example hook that we know
_does_ freeze its input and return a frozen value, so i've temporarily added
`useFreeze()` to the list of defined hooks. That is meant as a stopgap: the
right solution is to allow some way to tell the compiler about specific custom
hooks and their semantics.
Adds a subclass of ReactiveFunctionVisitor, ReactiveFunctionTransform, which
makes it easier to write passes that change the shape of a ReactiveFunction. The
two use-cases converted so far are both flattening away certain categories of
reactive scopes — this will make it easier to add a similar pass to prune scopes
that contain hook calls.
Effect.Capture is very similar to Effect.Read, but the only difference is that
this reference is stored somewhere via a Effect.Store.
Previously, any operand associated with a Effect.Store in the same instruction
would get aliased -- so there was no need to explicitly differentiate between a
"normal read" and "read that gets stored".
This difference is now explicit with FunctionExpression where every dependency
is "read" but only a few are "captured" for store (and mutation). In a follow up
PR, the mutating deps will have an Effect.Capture to differentiate from the
other non mutating deps (Effect.Read).
This commit adds a new Program visitor to our Babel plugin which then calls our
FunctionDeclaration visitor. Babel does some "smart" merging of plugin passes so
so even if plugin A is inserted prior to plugin B, if A does not have a Program
visitor and B does, B will run first.
Note that we also can't use Forget inside of a Babel preset as plugins run
_before_ presets (https://babeljs.io/docs/en/plugins/#plugin-ordering).
When this flag is enabled, Forget will only compile function declarations opted
in via the `'use forget'` directive. By default this is false.
Tested internally, see related diffs
Improves DCE, using fixpoint iteration to detect values that are updated across
loops but otherwise never read. There is still some further optimization we can
do (the dce-loop case could optimize out `y`), but this seems like plenty for
now.
Actually, we probably have to add some return statements to our fixtures before
landing this, because otherwise most of the code goes away.
This is a first pass at DCE without having read any literate on the subject, so
lemme know if there's a better approach. That said the algorithm is:
* Keep a `Set<Identifier>` of identifiers that are used (and whose constructing
logic cannot be removed).
* Do a first RPO iteration of all block's phis. Any phi operand that
participates in a loop is preemptively marked as "used" even if it isn't
strictly used somewhere. This step is necessary bc these operands may otherwise
not be used.
* Do a second post-order iteration of all blocks, including iterating first
their terminals, then reverse iteration of instructions, then their phis. Mark
the operands of each as used as we encounter them, and prune instructions whose
lvalue is never used.
For now I was conservative about which types of instructions can be pruned. For
example, call instructions are never pruned, even if the result of the call is
never used.
However one catch is that we currently prune instructions that cause values to
become frozen. We had planned to add runtime calls (in dev) to freeze values for
runtime enforcement, and if we want to do that we can always add these
instructions back (or replace them with explicit freeze calls).
There are a few potential next steps but we should discuss whether they're worth
it:
* Use fixpoint iteration to find exactly which operands are actually used. This
would allow us to to prune cases such as `let x = 0; while (...) { x += 1 }` eg
where there's a phi but the result is never used. Such cases should be rare in
practice though.
* Eliminate more types of instructions, eg eliminate function calls that don't
have any mutable arguments.
There's a bug in the HIR->ReactiveFunction conversion for certain categories of
compound value blocks where we replace operands (which must be a Place) with a
ReactiveValue. This approach worked in practice for lots of cases so I thought a
type coercion was safe, but then I found a case where this assumption breaks
(see new test).
The updated logic fixes the bug and is simpler. When a value block gets split up
(because there was a nested value block), instead of replacing the earlier value
in the later instructions, we append the instructions together. This can result
in some extra nesting (which if we wanted we could flatten away) but ensures
that we maintain type-safety.
A SequenceExpression currently doesn't store the InstructionId that produced its
final `.value`. This PR adds that instruction id, which is then used in the next
PR as we compose SequenceExpressions.
Just small things I noticed when looking at InferTypes. I was thinking about how
we'd adjust this pass to account for hooks, i'll probably pause that for now but
putting this up in case you like the changes. If not no big deal!
We currently lower switch case test values within the wrong scope: the test
value really should be a value block rather than a `Place`. Until then, this PR
adds a bailout for complex test values: we allow primitives and identifiers
which should cover most real-world use-cases.
No need for InferAliasForStores to know about the semantics of each instruction
anymore. It's just a simple pass that iterates over every operand and lvalue.
The FunctionExpression is special cased because it's slightly different but I
have a follow up that removes this special casing.
This doesn't change codegen as the lvalue is unused but it lets us make this
pass be semantically the same across all instructions -- "alias lvalue and
operands of an instr".
Adds limited support for UpdateExpressions (`x++`). We now support the postfix
form (`x++` ok, `++x` is a todo) and only when the argument is an identifier. We
can relax these restrictions with more work, but this PR should be sufficient
for the examples we've seen so far.
Support TypeCastExpressions — `(x: TypeAnnotation)`. This is pretty
straightforward, it's semantically identical to a raw identifier.
One catch is that our prettier config is hard-coded to use the babel-ts parser,
i wasn't sure how to make that dynamic based on the file extension so for now i
just ignored .flow.js files in our pretter config.
```
function useBar(props) {
let z;
if (props.a) {
if (props.b) {
z = baz();
}
}
return z;
}
```
Currently fails with
```
InvariantViolation: A phi cannot have two operands initialized before its
declaration
```
Changes ReactiveWhileTerminal’s test to use the new value block representation.
This means logical and condition expressions will work as while test values now.
Updates some passes from ReactiveScopes/ to use the visitor added in the
previous PR. The +124/-354 line count on this diff tells the story — the new
visitor avoids a lot of boilerplate and helps focus on the logic not the
traversal.
Note that there are a few passes which transform the function such as
adding/removing scopes. A follow-up will extend the visitor to support that and
convert the remaining passes.
This adds a truly general-purpose visitor pattern for ReactiveFunction, modeled
on what's worked well in Relay Compiler. All types of node that have children
get a visitFoo/traverseFoo pair of functions. By default the visitFoo() function
delegates to the traverseFoo() function, but the visit variant is meant to be
overridden and can delegate to the traverseFoo() variant — this gives you
precise control so that you can save/restore state before/after traversing
children.
Probably the only interesting thing is that visitLValue() does not call
visitPlace() by default though it technically could. So far that is making sense
in the passes i converted.
Note that once all passes are updated to use this, i'll delete the other visitor
helpers for ReactiveFunction.
There's a bug with assignment expression in normal value blocks due to LeaveSSA.
Until that's resolved i'm temporarily distinguishing "loop" blocks and "value"
blocks, and disallowing assignment expressions in value blocks specifically.
Support conditional expressions from AST -> HIR -> ReactiveFunction -> AST. This
also helps make the patterns for value block handling more clear, so i was able
to extract some reusable logic in the HIR -> ReactiveFunction conversion phase.
Implements the conversion from LogicalTerminal into a ReactiveLogicalValue (and
ReactiveSequenveValue if necessary). The implementation is a bit rough, i clean
it up in subsequent PRs which revealed parts of the logic that could be shared w
ternaries.
Extends ReactiveInstruction's value type to be a regular InstructionValue *or* a
LogicalValue. LogicalValue is operator, left, and right. It's really convenient
that we've already distinguished Instruction/ReactiveInstruction now — while the
_helpers_ here are updated to handle this new value type, the types ensure that
HIR can never encounter a LogicalValue.
The actual conversion of logical terminals into this value is complex and is
later in the stack.
Changes the lowering for LogicalExpression to use the new 'logical' terminal.
Whereas before we tried to more directly model the semantics of `??` by
generating an `if (<lhs> != null)`, we now generate a branch terminal that looks
at the lhs.
The HIR -> ReactiveFunction construction for logical and branch terminals are
placeholders while I refactor the ReactiveFunction representation to support
value blocks.
This moves the bailout recording mechanism into a separate CompilerErrors class
instead of repurposing HIRBuilder. This is to allow other passes to also record
errors instead of immediately throwing.
Previously we were storing a pointer to the HIR or ReactiveFunction
prior to printing, so when we printed them it would always print the
results of the last pass. This commit changes it so we print them to
strings when iterating through the compiler pipeline so each snapshot is
correctly preserved
This was causing issues in various places where errors would be stringified.
Because the inner detail objects would contain a NodePath with circular
structures this would cause a JSON.stringify error in code outside of our
control. This change makes it so we always print the codeframe from the NodePath
and then passing the string.
We need to know the kind of each block (regular or value). Rather than specify
the kind when closing the block — when we've lost context about why the block
was created — it's simpler and more accurate to specify the kind when
creating/reserving the block.
Previously, `yarn prettier` didn't write changed files since `glob` produced
paths relative to cwd and `git diff --name-only` produced paths relative to git
root directory.
This changes `git diff --name-only` to `git diff --name-only --relative`
(Implemented as per discussions with @gsathya )
Handle OptionalMemberExpression by adding an 'optional' flag to `PropertyLoad`
instruction, which is set during `BuildHIR` and read during
`CodegenReactiveFunction`.
This commit repurposes CompilerError to represent an aggregate of error details
accumulated during HIR lowering. It also fixes the playground to correctly
render errors again.
Functions can capture variables declared after the definition of the function.
This re runs SSA to map the captured identifiers to the new SSA identifiers if
available.
As discussed, this repurposes OtherStatement as a catch all variant for
unsupported syntax or errors in the source. This also renames the previously
added ErrorTerminal to UnsupportedTerminal for consistency (plus makes it a
little bit less confusing that it's not an actual terminal representing an
Error).
Not loving the name but couldn't think of anything better, open to suggestions!
When `MergeOverlappingReactiveScopes` identified scopes to be merged, it was
currently (oops, my bad) updating the _existing_ scope's id and range. Later,
PropagateScopeDependencies marks outputs of a scope by updating that
identifier's scope instance — so if that scope instance isn't shared, then the
output is lost. This PR fixes MergeOverlappingReactiveScopes to correctly update
all operands for a scope to have the same scope instance.
Makes debugging a little easier as the previous console.error would be logged
out of band with the jest error message. And the jest error would be missing the
error stack.
After some painful debugging I isolated the infinite loop when attempting to use
the BabelPlugin in hir-test rather than manually parsing and traversing it. The
issue is that in the BabelPlugin we were replacing the original
FunctionDeclaration with a new one, which would add it to Babel's traversal
queue. This would effectively create an infinite loop where we would try to
optimize a function that was already compiled by Forget (aside: _should_ running
the compiler multiple times on code work?).
To get around this we can just call the handy `skip` method on the new
FunctionDeclaration to tell Babel to stop traversing it. I'm also moving the
scope check here because I'll remove it from hir-test in a later commit.
Lower member expression if the receiver is in scope. Skip the remaining path
before capturing so we don't recurse down the identifiers in the member
expression.
Avoids printing debug information if it exactly matches what was last printed.
This means when debug printing (eg with `@only`) you'll see things like:
```
BuildReactiveFunctions:
...debug view...
FlattenReactiveLoops: (no change)
PropagateScopeDependencies:
...debug view...
```
Which saves time figuring out if something changed in a given pass.
Per design discussion, this PR changes BuildHIR to maintain the invariant that,
for each distinct variable in the input, that all references to that variable in
the HIR will have the same unique `name` _and_ same unique `id`. Phrased
differently: Identifiers with the same id will have the same name and
vice-versa.
This isn't an invariant we maintain throughout compilation — SSA form changes
the `id`s — but crucially, ensuring that the `name` is also unique allows us to
understand later which identifiers referred to the same original variable and
which were different.
Follow-up PRs will ensure that we maintain variable identifiers in the output as
well, in all cases except shadowing (and for shadowing, we'll rewrite
identifiers inside lambdas).
Adds `PropertyCall` and `ComputedCall`, which are a combination of
CallExpression and PropertyLoad/ComputedLoad, respectively. The goal is to
ensure that we correctly model the receiver of a call where the callee is a
member expression, and also accurately record scope dependencies in for both the
computed and non-computed (property) cases.
An alternative that I tried first was to add a `receiver: Place | null` to
CallExpression. That works well for HIR construction, but it's then very
difficult at codegen time to correctly reconstruct the original call: if the
receiver and callee share part of their structure then we can transform back to
a non-computed member expression, otherwise it has to be computed. Eg we have to
distinguish `a.b.c[d.foo]()` from `a.b.c[a.b.c.foo]()`. Given that our target is
high-level code, it seems reasonable to have a higher-level representation for
these cases.
I'm open to feedback but this feels pretty reasonable in terms of complexity /
precision of modeling.
Previously when converting from HIR -> ReactiveFunction we elided break/continue
terminals in places where control would implicitly transfer to the
break/continue target and therefore nothing has to be emitted. The one downside
of this approach is that it makes scope analysis a bit trickier. We want to
close scopes once we see an instruction id past the end of the scope's range,
but these implicit breaks were causing us to miss some instruction ids. We
compensated for this, but it's helpful to keep the representation explicit and
discard these terminals later in codegen.
Collapses HIRTReeVisitor into BuildReactiveFunction, allowing us to remove the
generic interface and simplify the code. Note that because the visitor was
already not attempting to group instructions by scope anymore, the visitor code
was very straightforward. This is mostly replacing calls to `appendBlock(block,
instr)` with `block.push(instr)`.
Cleans up the public exports for the package itself:
* `parseFunctions()` was only used in playground, so this moves the definition
there. I had to update playground's dependencies to ensure the babel version
matched.
* Flattens away the `HIR` const in the export, and exports just 4 functions all
at the top-level: `run()`, `compile()`, `printHIR()`, and
`printReactiveFunction()`.
This will make it very easy to split the core compiler into a separate package
from the babel plugin, though i'm not sure it's worth doing that (yet).
Other than BuildReactiveFunction (HIR -> ReactiveFunction), Codegen.ts was the
only other remaining place that we use HIRTreeVisitor. However, we've already
switched the compiler to use the new form of codegen,
CodegenReactiveFunction.ts. This PR extracts the shared code from Codegen.ts
into the latter, and deletes the unused bits of Codegen.ts which relied on the
visitor API.
This now frees us up to merge BuildReactiveFunction and HIRTreeVisitor, removing
all the complexity of the visitor trait and type params.
Note that the `ReactiveFunction` data type can represent non-reactive functions.
So we can still compile non-React code after this change, the pipeline is `AST`
-> (BuildHIR) -> `HIR` -> (BuildReactiveFunction) -> `ReactiveFunction` ->
(CodegenReactiveFunction) -> `AST`. Which is the exact sequence I mapped out at
the start of this project ;-) (happy that worked out!)
See the background in #982. This PR reimplements part of InferReactiveScopes,
merging overlapping reactive scopes, but against ReactiveFunction instead of the
HIR.
See the background in #982. This PR reimplements part of InferReactiveScopes,
aligning reactive scopes to block boundaries, but against ReactiveFunction
instead of the HIR.
The primary goal of this stack is to change HIRTreeVisitor to make it easier to
handle value blocks. That's complicated by the fact that the visitor is a
general-purpose visitor, used in several analysis passes including
BuildReactiveFunction (which translates HIR->ReactiveFunction while also
grouping instructions into scopes) and InferReactiveScopes (which is actually
two passes, one to align scopes to block boundaries, one to merge overlapping
scopes). The long-term goal then is as follows:
1. Make BuildReactiveFunction transform HIR->ReactiveFunction but _without_
reactive scopes.
2. Align scopes to block boundaries, but rewritten to operate on
ReactiveFunction
3. Merge overlapping scopes, again rewritten to operate on ReactiveFunction
4. Group statements within ReactiveFunction into ReactiveScopeBlocks (today this
occurs when constructing the ReactiveFunction).
This PR implements 1 and 4. Because the implementation is incomplete this would
break the whole compiler, so for now both versions are still around. By default
compilation uses the old pipeline, but if a feature flag is enabled we use the
new version. The plan is to incrementally fix up the new version of the passes
in this stack, and then cutover: removing the flag and the old version of the
passes.
It's not enough to only update the mutable range of the canonical id created
instead of the phi but we need to update the mutable range of each of the
operands of the phi as well to account for the fact that the phi could've been
mutated later.
The operands are updated only if the phi is mutated later. Otherwise these
operands can be cached in their blocks.
Fixes https://github.com/facebook/react-forget/issues/978
Changes playground to use the modified `run()` function of the compiler, polling
the generator and building up a Map of tabs automatically based on the passes
that the compiler runs. This means tabs are always derived from the current
state of the compiler and we can never forget to add a pass.
<img width="1497" alt="Screen Shot 2023-01-10 at 11 06 03 AM"
src="https://user-images.githubusercontent.com/6425824/211639442-da421f73-e19e-4b63-9f33-0ce5a68cceb7.png">
Note the inclusion of some recently added passes that weren't added to the
playground — which was my fault but only bc i intended to ship this PR soon :-)
Turns CompilerPipeline into two functions:
* `run()` is a generator and yields values that are a disjoint union of either
AST/HIR/ReactiveFunction along with a name for that step. The idea is to use
this in the playground so that it always matches the exact steps for
compilation. I'll update playground in a follow-up.
* `compile()` is ast in, ast out, and uses `run()` under the hood.
Implements constant propagation/constant folding for a conservative subset of
the language. The approach is described in detail in the comments in the file
itself, a key note here is that this pass currently emits what looks like
garbage:
```
// input
const x = 1;
const y = x + 1;
// output
const x = 1;
2; // <---- you'll see a bunch of lines like this
const y = 2;
```
These useless lines occur where previously there was a temporary getting
calculated that was used later (so we saved it until it was used), but now it
isn't used later so we just emit it in-place. Dead code elimination (DCE) can
eliminate these and other useless statements later.
Note that a key motivation for implementing this pass is to reduce memoization
blocks to what is strictly required for dynamic computations. Why memoize at
runtime when we compute at build time?
When we convert a LabeledStatement to HIR we can end up emitting "consecutive"
blocks, ie where there are two blocks such that control flow will always go from
from one block to the other, with no other way to reach the second block but
through the first. Example:
```javascript
label: {
foo();
break label;
}
bar();
```
Converts to
```
bb0:
foo()
goto bb1:
bb1:
bar();
...
```
Ideally in this case we would merge these into a single block:
* When debugging, the extra goto makes it look like there is conditional control
flow when there isn't. If the code is consecutive it's easier to understand that
if it's a single block.
* Conversion from HIR -> AST relies on consecutive code all being in a single
block, so this breaks codegen (we never visit the goto target since all gotos
are assumed to be safe to convert to a break or continue).
This PR adds a failing test case, the next PR fixes it.
Phi operands and Block predecessors currently use a `BasicBlock` reference
rather than the BlockId. This diverges from other places (like terminals) where
we use an id and not a direct object reference. Especially since blocks may get
rewritten or pruned, it's a bit cleaner to use the block id in these places.
Changes `shrink()` and `reversePostorderBlocks()` to modify the HIR in-place
rather than return a new function, for consistency with all our other passes
which mutate in-place (for performance reasons).
shrink visits all the fallthroughs even if they are unreachable so this isn't
the right place to prune unreachable blocks.
This PR moves pruning into a separate pass.
Supports computed properties (as LHS and RHS) correctly. Previously we only
handled member expressions where the property was an identifier, and would
incorrect treat `a[b]` the same as `a.b`. Now we correctly distinguish these and
convert `a[b]` as an IndexLoad and `a.b` as a PropertyLoad. Similar for
assignment, `a[b] = c` is an IndexStore. For both IndexLoad and IndexStore we
lower the property to a Place first.
Implements support for array and object de-structuring in variable declarations
and assignment expressions. Note that the code currently makes the overly
optimistic assumption that the RHS is an array or object that can be safely
indexed into. The correct representation would instead treat the RHS as possibly
iterable, but we need to consider the appropriate representation. I think it's
worth landing a first optimistic pass and we can iterate forward, this helps
make it more clear what the ideal representation would have to be and should
make a bunch of examples work. It also allows us to experiment with
representations of, and handling for, scope dependencies that involve computed
property access.
Adds new types for IndexLoad/IndexStore (renamed later in the stack to
ComputedLoad/ComputedStore) which will be used to represent computed property
access/update. The actual lowering to use these is later in the stack.
After the previous PR to change the scope dependency representation,
`Place.memberPath` is now completely unused, this PR deletes that field and all
references. Rejoice!
This is a pre-req to deleting the `Place.memberPath` field. We no longer need
memberPath in the HIR now that we have PropertyStore and PropertyLoad. However,
scope dependencies use memberPath to track the precise fields that a computation
depends upon. This PR changes scopes dependencies to use a new
`ReactiveScopeDependency` type (Place + optional path), which allows the next PR
to remove `Place.memberPath`.
Fixes codegen for chained assignment expressions. Previously each intermediate
assignment would be generated independently _in addition_ to the final chained
expression being emitted. We now emit a single chained expression, almost
exactly matching the input except for expanding from `x += 1` into `x = x + 1`.
There are two key changes:
* Ensuring that assignment expressions always generate an lvalue, which is
necessary for alias analysis to kick in, since it relies on the effect of the
lvalue to know where to look for aliasing.
* The above makes codegen think the entire assignment expression value is a
temporary that can be emitted later, but that isn't true. The new
PruneTemporaryLValue pass nulls out lvalues that are never read later, ensuring
that codegen can eagerly emit the value instead of saving it as a temporary.
Converts assignment expressions where the lvalue is a MemberExpression to use
PropertyStore, rather than creating an lvalue with a member path. The net effect
is that lvalue will always have a null member path.
There are two main cases:
* `x.y = <value>`. We lower <value> to a Place, lower the object of the member
expression to a place (`object)` create a temporary Place for the result of the
assignment, and then create a `PropertyStore <object>, "<property>", <value>`.
* `x.y += <value>` (and similar update-in-place operands). We extract the object
of the member expression, read the current value via a PropertyLoad, compute the
updated value, and store it back with a PropertyStore (each of these goes into
its own temporary).
Adds a `InstructionValue::PropertyStore` variant and the minimal necessary
handling across the passes to get things to compile. Lowering is in the next PR.
…-realsies"
This reverts commit 3c8700cbc7502e56cca8d4bb77a08e1ee747c166.
I'm not sure how this happened but I accidentally hit up + enter on a previous
`ghstack land` command for a different stack and it landed this one.
Infuriating.
For unknown reasons I didn't have the energy to dig into, some Babel Error
objects can't be written to so despite formatting the error message, the
original one would still be used. To get around this I'm just constructing a
fake object with a `name` and `message` so the correctly formatted messages are
used.
This is an incremental step to removing `Place.memberPath`. This PR changes how
we handle MemberExpressions in rvalue position, converting to a new
`PropertyLoad` InstructionValue variant. Example:
```
let x = a.b;
x.y = b.c;
=>
Const tmp1 = PropertyLoad a, 'b';
Const x = Place tmp1;
Const tmp2 = PropertyLoad b, 'c';
Reassign x.y = tmp2
```
That we already recently made a chance to ensure that _if_ the lvalue is a
member expression, that we convert the RHS to a Place. So although `x.y = b.c`
could technically be lowered to a single instruction (with the`b.c` as a
PropertyLoad), we force this to a temporary to ensure that we can independently
memoize the RHS value.
The net result is that the following combinations are possible:
* `x = y`, lvalue identifier, rvalue identifier
* `x.y = y` lvalue member path, rvalue identifier
* `x = y.z` lvalue identifier, rvalue property load
As noted above, `x.y = a.b` no longer occurs (and there's an invariant for this
in one of the passes).
A follow-up PR will add a PropertyStore instruction so that we can remove member
paths in lvalue position too.
@josephsavona had the intuition that we were picking the wrong root in the
previous infinite loop test case, so the fix for this is to force a root to
always be picked. This works because `find` implements path compression (if a <-
b and b <- c, then we can just point a <- c to "flatten" the tree which makes
subsequent `find` operations more efficient since we don't have to follow the
ancestor chain each time), so we're forcing all those unions to pick one parent.
From some googling it looks like the traditional way to implement union is to
call `find` so this should be the "right" way to fix it (?).
I'm also adding a basic unit test for DisjointSet, I think we could revisit
later and see if property testing is worth it but for now I mainly wanted to
capture the regression test as a unit test.
Discovered this by accident modifying Joe's playground example. This seemed to
be triggered in inferReactiveScopeVariables when iterating over the DisjointSet
of scopeIdentifiers. In particular this test case contains a cycle and
DisjointSet.find would never terminate.
The github action was exceeding maximum allowed memory size because we were no
longer grouping messages correctly prior to formatting them in the script. I
think these were introduced when we integrated the Babel plugin into the
preprocessor. This PR strips out filenames from the message so they can be
grouped together again. Also added some light comments
Test plan: manually ran `scripts/test262.sh` and verified that the JSON was
grouped together correctly
Previously we weren't ever clearing the `dist` directory so there were likely
some vestigial files left over from previous builds that might have confused the
import path. This commit fixes the import path and also deletes the `dist`
directory on every build to ensure a clean slate.
Given that our type inference needs to be very conservative, there's not a lot
of benefit to having such fine grained type inference.
In the future, we can use type information from flow/ts for inference.
For now, we've decided to punt on super fine grained aliasing of fields (ie,
mutating x.y shouldn't mutate x.z or it's aliases).
This PR removes the code that tracks the aliases of each field. We can re-add
this when we revisit this functionality.
For the rest of the field aliasing, most of it has been replaced by
InferAliasForStores.
Splits handling of simple assignment updates (`=`) from update-assignments (`+=`
etc). This unblocks starting to support destructuring for the simple case in the
subsequent PR.
The test262 preprocessor was correctly running forget against all the functions
in each test file, but it was incorrectly transforming the file contents
overall. Previously we swapped the contents of the file for the transformed
output of the last function, now we use the babel plugin to rewrite functions in
place and keep the rest of the file intact.
Of course, the tests that failed before still fail. But when we fix them the
tests will work now.
Addressed a TODO from the previous PR. When we enter SSA form, when we rewrite
variable reassignments we currently change the identifier but leave the kind of
the lvalue alone; technically we should convert from Reassign to Const. After
doing that, it's easier to correctly update when we leave SSA form, we can
convert just a subset back into let/reassign (but leave most things alone as
const).
Reorders LeaveSSA so that it runs before we begin evaluating reactive scopes.
Note that reactive scopes must span the full construction of each variable — for
variables with a phi, this must span the declaration and all assignments of the
phi operands. And that's exactly what the new LeaveSSA does! LeaveSSA removes
phi nodes and ensure that all versions of a variable which flow into a phi have
been assigned a single canonical identifier (with an appropriate mutable range).
This PR includes this and some related changes:
* Reorders the pass
* Changes hir-test to print the final HIR, eg just prior to codegen
* Teaches LeaveSSA to update the mutable range of the canonical identifiers it
assigns, based on the min/max of the variables assigned.
Rather than special casing for field stores, look for Effect.Store to alias
fields.
In the future, this will be extended to other constructs like Array#push.
Effect.Store is exactly like Effect.Mutate, the only difference is that Store
aliases one into a another value.
There is no practical difference between Effect.Mutate and Effect.Store
currently.
Assignment expressions to a member path are a special case because they're the
only place where a value isn't assigned to a (possibly temporary) variable,
which is our unit of memoization. #901 demonstrated how this can lead to values
that can't be independently memoized:
```javascript
const x = {a: a}
x.y = [b, c]; // array recomputed w `x`, even if only `a` changed
```
This PR ensures that assignment expressions where the LHS is a member path lower
the RHS to a Place. That means the above example is handled as if you wrote:
```javascript
const x = {a: a};
const tmp1 = [b, c];
x.y = tmp1;
```
And we independently memoize the temporary.
Completes a todo for the `if` condition of scopes without any inputs. In this
case since there are no inputs we can check for changes, we check if the first
_output_ cache slot is set to the sentinel.
For this to work we need to ensure that all scopes have at least one output,
which isn't currently the case. Dead code can produce output-less sentinels. So
this PR also adds a pass to find scopes w/o any outputs and convert them to
regular blocks. We could in theory also just delete them, but for now let's be
more conservative. This is something we'd want to highlight as a diagnostic in
an IDE, though existing dead code linters would almost certainly find this case
too.
Remove our existing compiler flags since they were only being used for
enabling/disabling passes to aid debugging and to simplify in preparation for
the upcoming work on diagnostics and bailouts. Additionally with the new
playground tabs disabling passes has become less necessary. In the future when
we have actual compiler flags (eg tweaking optimization levels) we can add this
back.
I opted to keep the existing `CompilerResult` return value instead of just
returning the optimized AST as we're still using `scopes` in our test fixtures.
Small reorganization to move Pipeline out of HIR since it's a compiler module.
It used to make sense before to be in HIR since the old architecture was the
still the primary, but no longer!
It's not safe to infer types of arguments and return values because Javascript
is so polymorphic. Instead just infer the type of the callee for non methods.
Interestingly, even this is not conservative enough for JavaScript because Proxy
can also be callable. But I think for our use cases we will treat Proxy and
Functions similarly (they're all just objects) so it's ok.
Thanks old architecture! We learned a lot about what does and doesn't work via
this POC, but it's time to move forward w the ~~new~~ current architecture.
When collecting the output of each scope I was checking to see whether each
operand was being used after the end of the _current_ scope. What we really want
to be checking is whether the operand is used after _the scope in which it's
defined_. Those are the same thing when there is no nesting, involved, so the
previous logic worked for most examples.
It isn't super easy to tell when if the operand's scope has ended, because we
don't always know what the "current" InstructionId is inside a ReactiveFunction.
And that in turn isn't quite so easy to change, because of some edge cases like
break statements that we synthesize. The solution here is to track the set of
active scopes, and if an operand is used and its scope is not active then voila,
it's scope must have completed and its an output.
Moves _some_ files from HIR into new top-level directories. To not make
@gsathya's life a pain I left the files he's touching alone, but I moved some
others. My intent is to have something like this:
* Babel/ - code for the Babel plugin, though ideally this actually gets split
into a separate package and the compiler itself is AST in, AST out w no Babel
dep.
* HIR/ - the core HIRFunction, HIR and related data types, plus the HIR
construction and printing, HIR visitors.
* Inference/ - the core inference passes that operate on the HIR, including type
inference, reference effects, alias analysis, mutable range analysis, etc. I'll
let @gsathya move these files when at a good stopping point.
* ReactiveScopes/ - inference relating to reactive scopes,
constructing/printing/codegenning ReactiveFunction
* SSA/ - enter/leave SSA and eliminate redundant phis
* Utils/ - every project needs a place to put stuff that doesn't fit into the
other categories, this is ours.
This leaves just index.ts at the top level, and overall feels pretty tidy. Not
too tedious to figure out where anything goes, hopefully.
It's pretty tedious to keep the playground in sync w `Pipeline` — we need some
abstraction so we can write the sequence of passes once and reuse it (while
inspecting intermediate states).
This is mostly unused and there's just not enough benefit for now. We can re-add
this if folks start using this site on mobile. Removing now to simplify the
codebase.
Updates the ReactiveFunction-based codegen from the previous PR to emit
memoization code for each scope. This is currently naive and has some bugs, but
it gets the idea across. The core logic is straightforward at this point, all
the hard work is in earlier passes:
* Compute one change variable per scope dependency, eg `const c_0 = $[0] ===
maxItems`
* Generate one `let` binding for each scope output
* Generate an if block where the test is if any of the change variables are true
(`||` them together)
* Generate the consequent block with the original code block, plus statements to
save dependencies and outputs to their cache slots
* Generate the alternate block to populate the scope outputs from their cached
values
## Todos
A few things don't quite work yet:
* Codegen is designed to avoid emitting variables for temporary values, but
that's causing a few values to sort of disappear n the examples, or get emitted
twice. There are a variety of ways to achieve this but we'll need to ensure that
this category of values gets assigned to a variable and then reference the
variable. This is more involved.
* Scopes can end up with zero dependencies, in which case we should check that
the first output cache is initialized. This one is more straightforward.
* If there are early returns, we don't record that they occurred and replay them
in the `else` branch for each scope. We know the algorithm though so i'm okay
delaying that for now.
Currently codegen operates from HIR using a tree visitor, but for scope
construction we're converting the HIR (CFG) into a ReactiveFunction (AST-like).
Our original idea for codegen was that we would convert the ReactiveFunction
back to HIR, and then codegen from there. However, the ReactiveFunction is
already in tree form...which makes it very straightforward to generate code
from.
So this PR implements codegen from ReactiveFunction. The output is _identical_
thanks in large part to reusing as much logic from Codegen.ts as possible. The
next PR will add memoization logic.
While we're collecting scope dependencies, we have the exact right information
to record scope outputs. These are variables that need to be defined outside of
the scope and populated by recomputing (on change) or via the cached value (if
no change).
I realized that properly propagating scope dependencies requires reusing the
same logic as dependency collection itself: a dependency of an inner scope
should only be propagated upward if the dependency was declared before the outer
scope, for example. So this PR reimplements dependency collection in the
propagation pass.
At the same time I made a few other improvements:
* Don't report dependencies that are "constant". This is a bit simplistic for
now, we can use a more advanced analysis later.
* Try to avoid creating duplicate dependencies.
This addresses the todo from the previous PR (flattening scopes in loops) since
now we don't need to compute deps until after that runs. As a follow-up i'll
remove the existing dependency collection.
Dependency collection has to visit the instruction id first before evaluating
the instruction, in order to completely any scopes that would end at that
instruction. Note the removed dependencies that don't appear within the scopes.
We can't independently memoize values created within a loop, so this pass
flattens scopes within loops. Right now this just flattens the scope away
without propagating any dependency (or output) information, follow-ups will
extend it to do that.
We don't need to create scopes for primitive values that are never reassigned.
The actual rules are more complex — we could choose to skip creating scopes for
values that don't allocate — but this simple heuristic is good for now.
The new LeaveSSA looks ahead to the phis of fallback blocks. However, HIR can
sometimes have multiple blocks with the same fallthrough (totally fine), so this
diff clears the phis of fallbacks as they are reached to avoid reprocessing
them. This caused a previously incorrect case to now fail, yay.
There are a bunch of ways we can go about converting from the input HIR into a
final form that has the preamble inserted and memoized blocks of code wrapped
with change detection and caching. This is just one way, it might not be the
ideal way. In any case, this pass converts HIRFunction -> ReactiveFunction. The
latter is a recursive (tree-shaped) data structure that attempts to represent
blocks each of composed of scopes or instructions, where scopes are themselves
composed of blocks etc. The idea is a) this makes it easy to visualize the
structure and check that the scopes and their dependencies are correct and b)
this is a really nice form for adding the memoization code. We can convert from
a ReactiveFunction back to an HIRFunction, wrapping each scope in the
appropriate if checks and caching.
## Example
Consider the following example, which has 2 main scopes: an outer one for `x`
and an inner one in the consequent for `y`:
```javascript
function foo(a, b, c) {
const x = [];
if (a) {
const y = [];
y.push(b);
x.push(<div>{y}</div>);
} else {
x.push(c);
}
return x;
}
```
## Output
The new builder constructs a ReactiveFunction for this example along the lines
of the following (note that inputs are always empty bc we don't collect those
yet):
```
{
scope @0 [1:11] inputs=[] {
[1] Const mutate x$11_@0[1:11] = Array []
[2] if (read a$8) {
scope @1 [3:5] inputs=[] {
[3] Const mutate y$12_@1[3:5] = Array []
[4] Call mutate y$12_@1.push(read b$9)
}
scope @2 [5:6] inputs=[] {
[5] Const mutate $13_@2 = "div"
}
scope @3 [6:7] inputs=[] {
[6] Const mutate $14_@3 = JSX <read $13_@2>{freeze y$12_@1}</read $13_@2>
}
[7] Call mutate x$11_@0.push(read $14_@3)
} else {
[9] Call mutate x$11_@0.push(read c$10)
}
}
[10] return x$11;
}
```
This shows the hierarchy: there's an outer scope, `@0` to compute `x` (the first
scope), then within the if consequent there's another scope, `@1`, to compute
`y`. We have some technically extraneous scopes to compute the JSX element; that
can be cleaned up with a bit more refinement.
With this structure — and the inputs and outputs of each scope filled in — we
can convert to code in a straightforward manner. Each scope turns into a block
along the lines of the following:
(note here we use strings to index the cache, in reality these would be ints)
```javascript
// one change variable pet input:
let c_a = a !== $['a'];
...
// one variable for each output:
let x;
...
// if (changed) { recompute } else { use-cache }
if (c_a || ... ) {
x = ...;
// one assignment per output
$['x'] = x;
// update cache per input
$['a'] = a;
...
} else {
// one assignment per output
x = $['x'];
...
}
```
TreeVisitor didn't distinguish between the type of a block and the type of an
item that can occur within a block - this was fine for Codegen which can use
`t.Statement` for both of those values. However, the upcoming scope construction
needs to distinguish instructions in a block from a block itself, so this PR
adds a new type parameter.
Per the title, this PR adds support for assignment expressions in update
clauses. This was mostly fixed by the previous diff to improve value block
handling, and there's only a bit more to do here to allow a "value block" that
doesn't produce a value (we need a better name).
This is a pre-req to construct reactive scopes in #857. "Value blocks" such as
`for` init/test/update and `while` test need to be consistently wrapped in
enter/leave calls so that we can extend the range of values properly. We also
need to handle `for` init blocks a bit differently, since they allow variable
declarations but not other types of statements.
This PR ensures that we use consistent methods for handling value blocks (`for`
test/update and `while` test) and treats `for` init as a new type with its own
enter/append/leave visitor functions.
Previously, this step just set the mutable range of any alias set including any
mutation to the end of the last mutable range of any of the containing
identifiers.
This change makes it so that the ends are only updated of the ranges that end
before the last mutation.
Fixes#852
Fixes https://github.com/facebook/react-forget/pull/858#discussion_r1044626137.
The case is
```javascript
function foo() {
let x$1 = 1;
let y = 2;
if (y === 2) {
x$2 = 3;
}
x$3 = phi(x$1, x$2)
if (y === 3) {
x4 = 5;
}
x$5 = phi(x$3, x$4);
y = x$5;
}
```
What happens here is that there are two _sequential_ phis for `x`. Previously
when we encountered the second phi we would find that there is no `let`
declaration for the phi or any of its operands, and create a new one before the
second `if`. That's incorrect, these should all merge into a single `x`
declaration. We now look up the phi operands to see if they are part of a
previous phi, and merge them correctly.
Reorders LeaveSSA so that it runs before we begin evaluating reactive scopes.
Note that reactive scopes must span the full construction of each variable — for
variables with a phi, this must span the declaration and all assignments of the
phi operands. And that's exactly what the new LeaveSSA does! LeaveSSA removes
phi nodes and ensure that all versions of a variable which flow into a phi have
been assigned a single canonical identifier (with an appropriate mutable range).
This PR includes this and some related changes:
* Reorders the pass
* Changes hir-test to print the final HIR, eg just prior to codegen
* Teaches LeaveSSA to update the mutable range of the canonical identifiers it
assigns, based on the min/max of the variables assigned.
A wise Joe once said, "We only care about observed aliasing".
Instead of performing aliasing for fields and non fields together, split the
analysis to happen over separate passes. Similarly split inferring mutable
lifetimes pass for fields and non fields.
Now, we can identify the aliases of fields that are *not* mutated and only alias
the ones that do mutate.
The algorithm is roughly as follows:
1. Build the set of aliases for non fields
2. Infer mutable ranges for all instructions except aliasing fields
3. Infer mutable ranges for all aliased instructions based on the alias set
calculated in step 1 (this doesn't include aliasing fields)
4. Extend the set of aliases (calculated in step 1) to include fields only if
the field or the receiver is mutated after the aliasing, ie, if _mutability is
observed_.
5. Run infer mutable ranges again for all instructions including the fields that
were aliased in the previous step.
6. Run infer mutable ranges for all aliased instructions including the fields
that were aliased.
## Problem
The previous version of LeaveSSA used a very simple approach in which
identifiers stored their pre-ssa id, and LeaveSSA restored this id back. The
upside of this approach is that it's very simple and trivially correct (assuming
no reordering of code). The downside is that after running LeaveSSA we lose all
information about which versions of variable declarations are distinct, and
which might merge together in a phi. That information is really useful for scope
analysis! Consider this input (variables are numbered as they would be in SSA
form):
```javascript
function foo(a, b, c) {
let x$1 = null;
if (a) {
x$2 = b;
} else {
x$3 = c;
}
x$4 = phi(x$2, x$3);
return x$4;
```
The current LeaveSSA assigns all 4 variables back to `x$1`, with a single let
declaration at `let x$1 = null`. However, from a reactive scopes perspective,
there are really just 2 versions of x: the initial x$1 (defined and never used)
and then x$2, x$3, and x$4, which have to be merged into a single scope because
they are part of a phi. In other words, we can't independently compute x$2, x$3,
or x$4 - if any of their inputs changes, we have to redo all the computation.
However, the existing structure makes it difficult to figure out the correct
starting point for this scope — there is no initial `let` declaration that we
can refer to.
Instead, we can represent the program as follows after LeaveSSA, and then use
this form for scope analysis:
```javascript
function foo(a, b, c) {
const x$1 = null; // NOTE: rewritten to const
let x$2; // synthesized declaration to allow later reassignment
if (a) {
x$2 = b;
} else {
x$2 = c;
}
return x$2;
```
Note that there are only 2 versions of x, and we have synthesized a variable
declaration for x$2 at the appropriate scope. Our scope analysis can then
determine that the range of x$2 is from the declaration to the end of the if.
## Approach
This pass does two main rewrites:
* For variables that do *not* appear as a phi id or operand, it rewrites the
declaration to be `const`. You can see this above for x$1.
* For variables that *do* appear as a phi or operand, it synthesizes a new `let`
binding at the appropriate scope (ie, in the appropriate block), and updates all
other operands from the phi to use the same id for the variable. You can see
this above for x$2, x$3, and x$4.
Note that the let binding is generated at the narrowest scope possible. In this
example, we generate distinct let bindings for the other if and else branches:
```javascript
function foo(a, b, c) {
let x = null;
if (a) {
// we generate a `let x$2` here
if (b) {
x = 0; // becomes x$2
} else {
x = 1; // becomes x$2
}
x // becomes x$2
} else {
// we generate a `let x$3` here
if (c) {
x = 2; // becomes x$3
} else {
x = 3; // becomes x$3
}
x; // becomes x$3
}
}
```
Because the different x values from the outer if/else can never join in a phi,
we can treat them as independent variables and (re)compute them independently.
The algorithm works by iterating in reverse-postorder, and looking ahead at
fallthrough blocks to find phi nodes that may need a let declaration (see above
example of where these are generated). It also tracks variables which _don't_
participate in a phi so that it can rewrite their declarations to `const`.
## TODO
This PR does *not* yet work for cases where there is unconditional assignment
within a `while` test condition. That would technically create a distinct
version of the variable that shadows the value for the loop, and you can't have
variable declarations in a while test condition.
That case already doesn't work, though, so i'm punting on it for now until we
figure out a bit more around
"value" blocks. We have some good options, like desugaring to a `for(;;)` and
manually implementing the while semantics in that case.
The changes here are pretty significant and there's a bunch more left:
- support for with any of `<init>`, `<test>` or `<update>` empty. - support for
with `<init>` as `Expression` instead of VariableDeclaration` node - support
assignment expressions in `<update>`, this seems like it might require further
new abstractions to allow something like a block to codegen into a single
expression.
The instructions of a while test node cannot just be pushed to the previous
block. This creates a new block for the test node and then during code gen
converts the statements pushed to the "value block" into expressions.
This adds a field sensitive, flow-insensitive, context-insensitive alias
analysis for lvalue aggregates.
In the future, InferMutableLifetimes can refine it's analysis using these field
sensitive alias sets.
Currently AbstractState.alias performs three operations: - Reading a value -
Storing the value - Updating aliasing (in the DisjointSet)
This commit makes AbstractState.alias only responsible for updating the alias
information. The rest of the operations are split into separate functions.
Moves the existing alias set building logic from inferMutableLifetimes to a
separate pass.
Additionally maintain abstract state to refine aliasing to not include
primitives.
HIRTreeVisitor previously passed a string label (for certain blocks). This
changes to pass the raw BlockId, and have codegen convert that to a string. I'm
not sure if we'll need this but it would be helpful for eg visiting the IR and
emitting a new IR, while mapping block ids forward. Even if we don't need that
it makes sense for Codegen to decide how to convert a block id into a label
(which has to obey the rules of an identifier, not the visitor's concern).
They're fairly related, but I figured it's worth keeping more examples.
- For the `while` example we need to codegen into a single expression. - For the
expression with contained assignment we need to either keep the SSA ids around
or re-create a similar expression during codegen.
This builds upon @josephsavona's prior PR #817 to add support for inferring
reactive scope dependencies for all instructions and terminals.
Still TODO (probably in follow up PRs):
- [ ] fix duplicate/different identity scope issue (see this [test
fixture](5f3b260aaa/forget/src/__tests__/fixtures/hir/overlapping-scopes-while.expect.md))
- [ ] also collect outputs of each scope - [ ] add new tree visitor that only
visits, consider renaming the current `visitTree` to `mapTree` or similar
Co-authored-by: Joe Savona <joesavonafb.com>
The `automaticLayout` config option for Monaco causes it to remeasure itself,
and the parent container's height being longer than the screen causes it to
constantly grow infinitely. You can observe this bug by going to the playground,
expanding any tab, and watch the scrollbar grow as the editor quickly grows to
ridiculous heights and your laptop starts glowing red hot and̶͙̕ H̴͉͘e comes
t̵͙́o ̴̜̿de̷̼̚s̷̻̍ec̶̮͒rate all knowled̵̥̆ge
The argument was unused and a confusing boolean argument that's easy to mix up.
Suggesting to remove it until we see a need for it at which point we might want
to introduce an enum to make the argument more obvious.
This is a follow-up to merging ranges. I realized that we need to mark terminal
ids as visited _before_ processing the branches of that terminal (whereas before
we were marking terminal ids only _after_ processing the branches). That exposed
another bug where interleaving could fail to be detected (with an error,
thankfully) if one of the branches had completed already.
Our small test suite is already really good!
This demonstrates a situation we don't handle well today. The basic structure is
that you have some variable defined at the top level, then some control flow
like if/switch where _all_ branches reassign the variable, then some code after
that references the resulting phi node:
```javascript
let x1;
// ... mutate/read x1
if (cond) {
x2 = {};
} else {
x3 = {};
}
x4 = phi(x2, x3);
```
We currently group x3, x3, and x4 into a scope together, but note that...there's
no `let` declaration for any of those! This means that it looks like the scope
for x2 and x3 start in the consequent/alternate, but the true scope spans from
before-after the if. I'm inclined to say that LeaveSSA should run _before_ scope
analysis, and produce something like the following in this case:
```javascript
let x1;
// ...mutate/read x1
let x2; // new variable declaration for the new version of x
if (cond) {
x2 = {};
} else {
x2 = {};
}
x2;
```
This then allows us to construct a correct range for x2, which starts in the
other block.
- [x] Merge scopes that are interleaved
- [x] Merge scopes if they both cross control-flow boundaries together
- [x] Don't merge scopes that strictly shadow
Still WIP because I want to double-check and see if i can find a simpler
algorithm for this. But it works.
This is a random driveby improvement. I realized that we can eliminate `return`
statements if a) they have no value and b) they are in the top-level block.
Functions implicitly return at that point — there can't be any succeeding
instructions anyway — so we can save bytes in the output.
Visits the HIR as a tree and updates mutable ranges to ensure their range end is
aligned with the block in which the scope is declared:
```javascript
function foo(cond, a) {
⌵ original scope
⌵ expanded scope
const x = []; ⌝ ⌝
if (cond) { ⎮ ⎮
... ⎮ ⎮
x.push(a); ⌟ ⎮
... ⎮
} ⎮
... ⌟
}
```
The implementation tracks the block in which each scope "starts" (first
instruction with an operand in that scope), and then finds the first instruction
at that block (or a parent) which is after the scope's end.
Refactors `Identifier.scope` to be a `ReactiveScope` object with an id and
range. This gives us a place to later add a list of dependencies for the scope.
- Add a `loc` to the `while` terminal node.
- Move location data for assignments 1 level higher as that seems to work better
in the generated code (not tested with actual debugger yet though, we'll
probably want to look at this more closely.
Refactors Codegen to extract the core "visit IR as a tree" logic separately from
the code to emit JS:
* `HIRTreeVisitor` is a new helper that visits the HIR as a tree. You call
`visitTree(ir, yourVisitor)` and it drives visiting of the IR, tracking blocks
and scopes and calling methods as appropriate.
* `Codegen` is now implemented as a Visitor implementation. For example
`enterBlock()` creates an empty `Array<t.Statement>`, `leaveBlock()` wraps that
in a `t.BlockStatement`, etc.
* `printHIRTree()` is a new IR printer (implemented as a visitor) that prints
the HIR in tree form, so it retains the original shape of the code but with each
block replaced with its IR equivalent.
The new pretty printed scopes "syntax" breaks mermaid labels because the `@`
character seems to be reserved. This wraps them all as a string so they work
again.
Expands InferReactiveScopeVariables to update the mutableRange of all
identifiers to be the range of its scope. The result is that all identifiers in
a given scope will have the same range, whose start is the minimum of the
identifiers range starts, and end is the maximum.
This completes the implementation of InferReactiveScopeVariables, adding support
for phi nodes. Example:
```javascript
let x$0 = null;
mutate(x$0);
if (cond) {
x$1 = a;
mutate(x$1)
} else {
x$2 = b;
}
x$3 = phi(x$1, x$2);
mutate(x$3);
```
We now add x$1, x$2, and x$3 to the same reactive scope. This reflects the fact
that x$3 cannot be computed without also computing both x$1 and x$2. Note that
x$3 can never be x$0, so x$0 is _not_ added to the same scope. This allows us to
take advantage of SSA form to note that _some_ instances of an identifier really
are distinct.
There's no option to output the results of the test262 harness in silent mode so
every pass and failure outputs multiple lines to stdout. Since there are many
thousands of tests this results in unusable log files that are over 200k lines
long. This PR redirects stdout to a tmp file and then we reformat the result
into a small JSON object, grouped by the failure message with count.
Example:
```json [ { "pass": false, "data": { "message": "Expected no
error, got Error: TODO: Support complex object assignment", "count": 66
} }, { "pass": false, "data": { "message": "Expected no
error, got Error: TODO: lowerExpression(FunctionExpression)", "count": 4
} }, { "pass": false, "data": { "message": "Expected no
error, got Error: TODO: lowerExpression(UnaryExpression)", "count": 6
} }, { "pass": false, "data": { "message": "Expected no error,
got Error: todo: lower initializer in ForStatement", "count": 28 }
}, { "pass": false, "data": { "message": "Expected no error, got
Invariant Violation: Expected value for identifier `15` to be initialized.",
"count": 14 } }, { "pass": false, "data": { "message":
"Expected no error, got Invariant Violation: `var` declarations are not
supported, use let or const", "count": 76 } }, { "pass": true,
"data": { "message": null, "count": 1 } } ] ```
Add a new workflow to run the test262 tests on commits to main but not in pull
requests. This is to keep this test non-blocking on PRs but lets us track pass
rates over time
Adds a new pass `InferReactiveScopeVariables` which determines the sets of
variables (by Identifier) which "construct together" and belong in the same
reactive scope. Concretely, `Identifier` gets a new property `scope: ScopeId`,
and this pass assigns each identifier a ScopeId value. The algorithm iterates
over all instructions in all blocks (in a single pass) and builds up disjoint
sets of identifiers that appear as mutable operands in the same instruction.
The algorithm is relatively simple (especially since I had already implemented a
union-find data structure): however looking at some examples reinforced that
other planned todos around alias analysis are really important. We also have to
think more about what "mutable lifetime" means in the context of SSA: currently
variables that are reassigned (but never "mutated", eg bc they're assigned a
value type) never appear as mutable.
Just realized we can run all tests without encountering the arg limit if a
string is passed in.
This is much better because the test runner will count all tests in the parent
test directory rather than run the tests in each subdirectory
Noticed this while running test262 tests that many variables were throwing an
invariant for being undefined. This includes things like the special `arguments`
object, a global `assert` function used by test262, etc.
- Adds a shallow git submodule for test262 as the tests aren't available as an
npm module - To run all tests: `yarn test262:all`. Note that this chunks up the
tests by test262 folder as there are over 50k+ tests and the test harness only
accepts arrays of filepaths which exceeds arg limits - To run a specific test:
`yarn test262 test262/test/folder/file.js`. You can also pass globs which
expand into an array of filepaths: `yarn test262 test262/test/folder/**/*.js` -
More instructions for the test-harness can be found here:
https://github.com/bterlson/test262-harness
I noticed on @kassens's #771 that despite running LeaveSSA there are still cases
where we still reassign to a unique identifier: functions that have reassignment
but no phi nodes, such as:
```javascript
function foo() {
let x$1 = 0;
x$2 = x$1 + 1;
}
```
Here SSA form rewrote the second statement's LHS, but bc there's no phi node we
can't recover what the original was supposed to be (`x = x + 1`). This was my
oversight when suggesting the simpler LeaveSSA algorithm, it works for
eliminating phis but not other reassignments. The only alternative to removing
SSA form is to add assignment statements, which we obviously don't want to do
since that generates bloat.
This PR addresses the issue by adding an additional, optional property to
`Identifier` called `preSsaId` that starts off null. When entering SSA we save
the original id in this property and update id to a new SSA value. LeaveSSA does
the inverse, setting id = preSsaId and nulling out the latter. This means that
an identifier can always be uniquely identified by its `id` value at any point
in the compiler, while it's trivial to correctly undo SSA form.
```typescript
type Identifier = {
// Unique value for each original identifier
id: IdentifierId;
// The original, un-mangled variable name if this was a variable present in the
source (null if it's generated)
name: string | null;
// When in SSA mode, this is set to the original, pre-SSA `id` value
preSsaId: IdentifierId | null;
}
```
Implements assignment expressions with operators other than `=` (such as `+=`)
by lowering to an assignment.
I think this isn't fully correct for something like `a.b.c += 1`, but it seems
like there's more gaps in object accesses.
- Get rid of `indent` as it was making the code hard to read - Remove
unnecessary 2nd iteration over blocks - Remove extra newline between the bb
subgraphs and the jumps section - Remove trailing spaces - Remove newlines
between each subgraph and jump
We might need to revisit the tabs vs. options split, but for now this just adds
a checkbox toggle that outputs codegenned JS instead of HIR in the HIR tab. Open
to ideas to organize this in the future...
This PR adds a new section to fixture tests, which renders the HIR into
a visualization using mermaid.js syntax which can then be embedded
directly into markdown.
The nice thing about the mermaid syntax is that it's quite readable, so
if desired we could replace the current basic block textual output with
the mermaid block. I'm opting to append it for now and wait for feedback
if we want to keep both or replace.
To view the graphs in your editor, download an extension that
adds mermaid.js support:
https://mermaid-js.github.io/mermaid/#/integrations?id=editor-plugins. In vscode
you can use this plugin by right clicking on "Open Preview" on any expect.md
file. No extra dependencies are required for GitHub which should have builtin
support for mermaid in markdown
Instead of hacking into the babel plugin passes, this now takes a new approach
for the HIR tab:
- The different passes are exported from the babel plugin (for simplicity)
- The tab actually runs the compiler steps based on local config in the tab.
- Re-purposed the CompilerFlags component to configure what passes to run. This
is currently mostly causing different errors, but could be useful going forward
as a direction.
The `--watch` argument forwarding wasn't working anymore because `yarn build`
doesn't have the `tsc` command at the end anymore. There's maybe something that
can be done to forward the watch argument, just call `tsc --watch` directly.
Small adjustment to the previous PR for a special case:
```javascript
while (cond) {
break;
}
```
The loop body is an indirection to the fallthrough, so shrink() collapses that
and makes the while.loop === while.fallthrough. We now detect that this is the
case in codegen and correctly emit a `break` rather than trying to write the
fallthrough block inside the loop.
Adds a new 'while' terminal variant, which will be a model for other loop
terminals, and adds support for the entire compilation pipeline through codegen.
To understand the structure of the terminal consider this input:
```javascript
let x = 0;
while (x) {
x = foo(x);
}
return x;
```
We currently lower this to ifs and gotos:
```
bb0: precursor to loop
let x = 0;
goto(break) bb1; // <-- **The new terminal replaces this**
bb1: test block, whether to (re-)enter the loop
if (x) consequent=bb2 alternate=bb3;
bb2: loop body
x = foo(x);
goto(continue) bb1;
bb3: fallthrough after the loop
return x
```
This representation correctly models the semantics of while statements, but
loses the high-level information that there was a loop. The new 'while' terminal
replaces the first 'goto(break) bb1'. Conceptually, the 'while' terminal means
"enter the starting point of a while loop". In this example the terminal would
look like this:
```
{
kind: 'while',
testBlock: 'bb1', // the basic block that checks whether to enter the loop or
not
loop: 'bb2', // the block containing the loop body
fallthrough: 'bb3' // the block that goes after the loop
}
```
Most passes will only look at 'testBlock', ie they will treat this terminal as a
simple goto:testBlock. However, codegen uses the full information in the
terminal to reconstruct the loop. My previous PR, #755, added a mechanism to be
smart about when to emit or not emit `break` statements; this PR improves upon
that to accurately emit the minimal break and continue statements: ie omitting
entirely where they are extraneous, emitting unlabeled break/continue when
sufficient, and falling back to labeled break/continue only where strictly
necessary. The logic is very much analogous to IR construction.
Alternative approach to #750. We now store the original identifier on the Phi
node, then rewrite every BasicBlock's identifiers to reference the original id
instead of the SSA'd id.
This solves the shadowing problem and also lets us omit adding copies of
instructions.
The approach is very similar to what BuildHIR does to resolve break and continue
targets during IR construction:
* We annotate goto targets as either a break or a continue (during HIR
construction). This is necessary to reconstruct the right kind in codegen.
* Codegen continues to work by traversing the IR as if it were a tree, relying
on the `fallthrough` branches of if/switch to be able to visit the
consequent/alternate recursively and then emit the fallthrough branch.
* We track a Set of blocks that are scheduled to be emitted by some parent in
the tree. Nested ifs may all have the same fallthrough branch, which we only
want to emit once. This set helps us to know that a parent is already going to
emit some block, such that children can skip it.
* We also keep a stack of break targets that are in scope, and use this to
convert gotos appropriately, as either a break, continue, or nothing at all (for
example a switch case that falls through has no explicit syntax to model this
fall-through, the only option is to emit nothing for the goto).
* Then, if/switch have to carefully check whether each branch should be emitted
or not. For example, if the alternate is already scheduled to be emitted (by a
parent), then we emit a block with a break statement instead.
* Switch in particular is tricky, because we need to know that subsequent cases
are scheduled, but only for preceding blocks. So we visit the cases in reverse
order (not surprisingly, we do the same thing during IR construction for similar
reasons!).
The bookkeeping is a bit finicky but this works reliably. There are some cases
where we could try to emit an unlabeled break instead of a labeled break, or
avoid emitting a label at all (if nothing will explicitly break to that label),
but overall the generated code is readable enough that i'm inclined to ship and
iterate. I'm open to feedback though, as always!
Reverts #726 which added an early optimization to the SSAify pass in skipping
over phi creation if only one unique operand. This is no longer necessary with
the addition of a phi elimination pass added in #739.
This is an alternate take on phi elimination to the one we pursued over VC w
@poteto driving. This version exploits the RPO ordering of blocks to do phi
elimination in a single pass when there are no loops, and to minimize repeated
visits when there are loops. The main difference is when redundant phis are
removed. Rather than eagerly walking through the CFG for each pruned phi to
rewrite its uses, we build up a mapping of rewritten identifiers. As we walk
through subsequent instructions, we rewrite each place based on that mapping. We
continue cycling through the blocks so long as a given iteration *both* added
new rewrites (meaning there may be subsequent uses to rewrite) *and* there are
back-edges. With no loops this results in a single visit of each block and of
each instruction, but even with loops this is bounded.
This diff adds styling to the compiler options editor. The floating input/output
toggle button on small screens now spans the bottom of the screen, so that it
doesn't block the compiler options.
Height overflows when adjusting screen size are complicated by Monaco Editor and
will be addressed in a later diff.
Test plan:
Start Playground and see the latest look of the compiler options editor beneath
the output section.
Removes all the `path: NodePath` values from various IR node types, replacing
them with `loc: SourceLocation`. This type is an alias for babel's source
location type plus a "generated" variant. The "OtherStatement" kind also used
the `path` to print back the original AST (since we don't look into these);
instead, we now capture the underlying `node` and emit that as-is during
codegen.
While I was doing this, i also fixed up the places where we had passed a null
path; the vast majority have a clear place we can pull a location from. For
example, `a ?? b` syntax creates some places/instructions for the `a != null`,
but those can all point back to the `a` location.
This commit introduces a small update to our fixture tests to allow certain jest
`test` modifiers to be added to fixture tests via a leading prefix in the
fixture name.
For example, `only.my-fixture.js` is the equivalent of writing
`test.only("my-fixture")`.
This currently basically lowers the code into the equivalent of
```
const vLeft = <left>;
const vNull = null;
const vCond = vLeft != vNull;
vCond ? vLeft : <right>
```
I created a temporary `Place` to hold the `null` constant value because the
binary operator in HIR accepts only `Place`s. Not sure if this is the preferred
approach. Alternatives I could think of:
- Allow constants as an alternative to Place?
- A `NotNull` operator for `<x> != null`
- Some other extension to the HIR?
## Proper Detection of Out-of-order Functions
The no-use-before-define rule from ESLint has a strange behavior in which it
treats variables differently than functions:
```javascript
function foo() {
return bar(X);
}
const X = null;
function bar(x) {}
```
By default, `bar(x)` has two errors: one because X is used before defined, and
once because `bar` is used before defined. The rule has an option `{variables:
false}` which only enables validation when the variable is from the same "scope"
as the reference, the net result of which is it means it doesn't report spurious
errors such as X being undefined. There is _also_ a `{functions: false}` option,
but for some reason that doesn't work the same way, it just turns off all
validation of references that came from functions. So enabling that option would
suppress the (spurious) error on invoking `bar()` above, but causes the rule to
miss invalid code such as:
```
function foo() {
return bar();
function bar() {}
}
```
This PR adds a fork of the rule that makes `{functions: false}` behave similarly
to `{variables: false}`, which should help avoid some of the spurious errors i
saw internally. The rule is exported from Forget itself, which will make it
easier to consume internally, in tests, and in the playground.
## Targeting the validation to Forget functions
Even with the above, there are still some false positives coming from code such
as:
```javascript
const x = foo();
function foo() {}
```
This PR changes codegen to ensure that the output of a function _always_ has the
body starting with 'use forget'. The ESLint rule then only looks at function
declarations/expressions whose body starts with that expression. The new unit
test confirms that the validation finds invalid reorderings even on functions
that weren't explicitly tagged as 'use forget'.
This just piggybacks on the infrastructure for handling Env.#variables.
In the future, a better approach would be to simplify the environment creation
and merging by leveraging the SSA property of the new IR -- 1) We don't need to
track IdentifierId per environment as they are all unique 2) Rather than
tracking values, we can just track Identifiers because Identifiers can never
be reassigned.
The semantics of lvalue changes based on whether lvalue.place.memberPath is null
or not. If it's null, then lvalue.place acts as the lvalue for the instruction,
otherwise it's just a reference to the memberPath specified location.
Ideally we'd have an MemberExpression IR that lowers this complex lvalue into a
temporary Place, uses this temporary place and stores back to the
MemberExpression.
Working around for now, will refactor to create a MemberExpression in the future
if necessary for other analysis.
Instead of using Place, use Identifier as the unit of comparison in SSA.
Place is too high level and can not be substituted for other Places (even those
with the same Identifier) as Place contain higher level metadata such as
memberPath.
Currently, HIR doesn't load global idenfifiers into a temporary Place which
means our SSA transform breaks when it tries to lookup this global identifier.
Instead of throwing, let's log and return the old place. This works for now but
will probably break when we start mutating globals, but at that point our HIR
builder will need fixes.
ESLint's default parser doesn't support any non-standard syntax, which includes
JSX. So when I added the ESLint validation step to the playground, it meant that
valid examples containing JSX still reported "invalid output". I tried to use an
alternative parser, but I couldn't figure out the right webpack incantations to
make `@babel/eslint-parser` or `hermes-eslint` work. I even tried recreating
some of their code to avoid problematic imports, no dice.
Instead this PR:
* No longer uses the `postCodegenValidator` step, and runs the validation on the
output after compilation completes. This is better anyway since we can see the
output *and* the error messages
* Shows rule violations as an "Invalid output" comment
* Shows parser errors as a note (mostly to indicate that the validation step
couldn't run, there could still be no-use-before-define violations that weren't
found)
Invalid example:
<img width="1502" alt="Screen Shot 2022-10-21 at 9 50 23 AM"
src="https://user-images.githubusercontent.com/6425824/197249007-1ec244a0-6dfe-4ec6-a0d0-60302efd86bd.png">
Sample example but with some JSX:
<img width="1500" alt="Screen Shot 2022-10-21 at 9 50 39 AM"
src="https://user-images.githubusercontent.com/6425824/197249030-e68ba968-4101-47c7-a148-f548f84f375c.png">
Imports the runtime from `React.unstable_ForgetRuntime` rather than from a
separate module. The hope is that any code that gets transformed already has a
dependency on React anyway, so we can avoid adding a new dependency that other
systems don't know about.
While here, i also cleaned up the `guardThrows` flag (we still parse it if
present and warn, rather than throwing, to make it easier to adopt the latest
version in various places).
#686 added an option to validate generated code after transformation and adds an
ESLint-based validator function to transform-test. Unfortunately it isn't super
easy to wire up ESLint for use in a browser: traditionally the ESLint project
specifically did _not_ support browser builds, but they recently have relaxed
this because they added a browser playground on their website. There isn't
official support, but the [playground
repo](f3b1f78cc1/webpack.config.js)
has a webpack config that, when combined with requiring a specific file, allows
making things work in a browser.
I tried using this directly in our playground app but Next's default webpack
config doesn't work. So I created a separate package, playground-validator,
which exports a webpack-built version of `eslint.Linter`. Then the playground
can consume that, and everything works:
## Test Plan 👀
Confirmed that a known problematic example displays the validation message in
playground (both locally and on the preview deployment):
<img width="1500" alt="Screen Shot 2022-10-20 at 12 22 59 PM"
src="https://user-images.githubusercontent.com/6425824/197041265-966ffda2-a3d0-450e-8fc4-fd1a7ca06e1a.png">
Adds a new compiler option `validateNoUseBeforeDefine`, which enables a
post-codegen pass to validate that there are no usages of values before they are
defined (which causes a ReferenceError at runtime). This can occur when a value
is accessed when its in the TDZ (temporary dead zone), after the hoisted
_declaration_ but before the variable is defined:
```javascript
function foo() {
x; // x is in the TDX here: the binding from the subsequent statement is
hoisted, but x is not yet defined.
let x;
}
```
* The validation is off by default, but enabled in transform-test
* The validation crashes compilation, rather than bailout, because the code has
already been mangled and we can't roll back at the point the validation runs.
* The validator uses ESLint's no-use-before-define rule by printing the program
to source and then configuring ESLint to use Hermes parser.
* transform-test now supports tests prefixed with "error." to indicate tests for
which compilation is expected to crash (not just bailout), and the expect file
includes the error message.
Some prior [microbenchmarking](https://jsbench.me/7ol98ws520/1) showed that a
for loop outperformed `fill` (which is about ~60% slower). This is the same
approach we use in the latest useMemoCache PR
This is a new module that holds:
- the `useMemoCache` stub (hopefully to be deleted next week)
- various helpers that can be imported by the compiler, e.g. the dispatcher
guard `$startLazy`
- skipped the implementation of `makeReadOnly` for now as there's already
multiple copies and I wanted to avoid typescript in this file for now to make
the build easier (i.e. no build)
I'm not sure why exactly but previously this diagnostic message was
unusually slow to typecheck. Lifting the getter for init outside of the
diagnostic to the callsite seems to fix the hotspot. Probably some
interaction with string interpolation, or something else.
Test case: ran `yarn ts:analyze-trace`, hotspot for Diagnostic.ts no
longer present
Previously the PassManager would console.error if an unexpected error was
thrown, to help with debugging jest. However because we now capture all
invariants in compiler passes as bailouts, these are already captured in fixture
tests.
Additionally, we also already console.error if we find an unexpected bailout in
a fixture test. So this is purely redundant and removing reduces some noise when
running tests.
The current allowlist for capitalized function identifiers only allows stdlib JS
modules. This PR introduces a new compiler option to allow passing in a set of
allowed capitalized user function identifiers. It's a hack (in the absence of a
type system) to let us check that capitalized function calls are only possible
for identifiers that aren't bound to a React component.
I considered adding a mechanism in fixtures to configure compiler options
per-fixture, but opted to keep it simple for now and special case a
`ReactForgetSecretInternals` identifier as one allowed user function in
transform tests.
The CompilerError module's `invariant` is wired up to our bailout system so in
compiler passes should be preferred to the raw `invariant` module.
We should probably add an internal eslint rule to suggest using CompilerError if
the file is in one of the compiler pass directories.
* Renames `Capability` to `Effect` and clarifies the kinds as Freeze, Read, and
Mutate. The real intent of what we're inferring/representing is "what effect
does this reference to the value have on its value". Ie freeze freezes the
value, mutate mutates it.
* Consolidates Capability and EffectKind into Effect
* Renames some properties on Place for clarity
* Adds `value: ValueKind` to Place, which indicates the (merged) kind of the
value at that place at that point in the program.
* Changes HIR printing to show the effect and the value kind
* Simplifies some inference logic
* Changes HIR to store blocks in reverse postorder, which allows forward data
flow analysis to iterate the blocks in order and (in the absence of loops) see
all predecessors before visiting a successor.
* Updates reference kind inference to exploit this ordering
Note that the approach of modifying the ordering in `mapTerminalSuccessors()`
feels gross, i'd like to split this up a bit.
Scaffolds out mutable lifetime inference and the potential two-pass approach,
with motivating examples. Also adds some fixtures that collectively demonstrate
a bunch of cases of aliasing:
* direct assignment `a = b`
* property assignment `a.b = b`
* array literals `a = [b]`
* object literals `a = {b}`
* mutable arguments to the same call `foo(mut a, mut b)`
* return values aliasing arguments `a = foo(mut b)`
* aliasing that occurs only after multiple loop iterations
All of these fixtures use an empty `if (varName) {}` as a way to check that an
otherwise readonly usage of a variable is correctly inferred as mutable.
This implements an alternative approach to reference kind inference in the new
architecture based on feedback. Here, we track an environment that maps
top-level identifiers (IdentifierId) to the "kind" of value stored: immutable,
mutable, frozen, or maybe-frozen. We then do a forward data flow analysis
updating this environment based on the semantics of each instruction combined
with the types of values present. For example a reference of a value in a
"mutable" position is inferred as readonly if the value is known to be frozen or
immutable. Similarly, a usage of a reference in a "freeze" position is inferred
as a freeze if the value is not yet definitively frozen, and inferred as
readonly if the value is already frozen.
When multiple control paths converge we merge the previous and new incoming
environments, and only reprocess the block if the environment changed relative
to the previous value. This has some noticeable benefits over the previous
version:
* We now infer precisely where `makeReadOnly()` calls need to be inserted, aka
points where a value needs to be frozen may not yet be frozen.
* We track immutable values and can infer their usage as readonly rather than
mutable.
* The system handles aliasing by representing values as distinct from variables,
so that we can handle situations such as:
```javascript
const a = []; // env: {a: value0; value0: mutable}
const b = a; // env: {a: value0, b: value0; value0: mutable}
freeze(a); // env: {a: value0, b: value0; value0: frozen}
mayMutate(b); // ordinarily inferred as a mutable reference, but we know its
readonly
```
I didn't make this an option as it's unclear we'll really need this. We can
always add an option later, I think.
This is the name that's available on facebook.com at the moment.
Control dep should only affect how things are invalidated, which are modeled as
defs including declarations, writable uses to variables and expressions.
Closes#633
commit-id:41bd6fe5
Inputs occured in depGraph cycle is dangenrous and should be treated as an
invariant since Forget _may_ generate broken code in this case, despite that
technically this is a stricter then what we needed for the particular case of
#633 and #634 and there could be case that this is safe (like many `cfg-`
tests that I have to mark as `bailout.`)
I expect the next diff will fix them though.
commit-id:0b13ed02
Distinguishes between `LValue` and `Place`. For the most part this is the same
data structure (LValue composes Place), but it's helpful to distinguish them
since LValue has other properties such as the kind of declaration. The
representations may diverge more in the future.
This change lets us correctly emit code for variable declarations: previously we
didn't emit `let` or `const`.
Flushes out basic codegen for switch statements. This is more indication that we
can recover nearly the original source even for complex control-flow, given the
right IR design.
NOTE: this improves the equivalent of "ref kind inference" in the new
architecture. I'd appreciate review here on the algorithm in particular, but in
general my plan is to try to implement this on the current architecture.
The previous InferMutability reference kind inference didn't properly handle
capturing or reassignment combined with control flow. This is a new version
(i'll clean up to delete InferMutability entirely) that is less ambitious but
fully accurate (i hope, hence WIP):
* Annotates all references of frozen variables as frozen. This includes
following reassignment, so if you do `const x = props.x; foo(x);` we know that
`x` is frozen because it derived from a frozen value. This even works
conditionally, so if `x` is conditionally assigned to some value derived from eg
props, and you later use `x`, that will be marked as frozen even if it could
have other values at runtime (since it must conservatively assume a frozen value
flowed in at runtime).
* Annotates references that may mutate as mutable.
* Annotates references that are not frozen, but not mutated _at this reference
site_, as readonly. It's possible that a readonly usage is followed by a mutable
usage, since we don't yet know the "lifetime" of the mutability.
The notable difference from the previous attempt is that we do not attempt to
find the point at which a formerly-mutable values becomes readonly (and
therefore eligible for caching). That requires pointer analysis, let's discuss
offline.
Var declarations were treated identical as other declarations which cause code
relying on them getting hoisted now triggers runtime exception on TDZ.
This diff fixed that by generating `var` for `var` so they can be hoisted as
usual.
commit-id:00ab02f6
* [hir] Core data types and lowering for new model
* Handle more expressions, including using babel for binding resolution
* test setup with pretty printing of ir
* Basic codegen and improved pretty printing
* avoid else block when if has no fallthrough
* emit function declarations with mapped name/params
* start of scope analysis
* saving state pre-run
* add slightly more complex example and flush out lowering/printing (jsx, new, variables)
* Various improvements:
* Convert logical expressions (|| and &&) to control flow, accounting
for lazy evaluation semantics.
* Handle expression statements
* Improve printing of HIR for unsupported node kinds
* Handle more cases of JSX by falling by to OtherStatement to wrap
subtrees at coarse granularity.
* improve HIR printing, lowering of expression statements
* handle object expression printing
* improve IR model for values/places along w codegen
* more test cases
* start of mutability inference
* passable but still incorrect mutability inference
* improved mutability inference, should cover most cases now
* visualization of reference graph
* correctly flow mutability backwards (have to actually set the capability)
* separate visualization in output
* consolidate on frozen/readonly/mutable capabilities
* cleanup
* conditional reassignment test (not quite working)
* hack to output svg files for debugging
* handle conditional reassignment
* improve capture analysis
* treat jsx as (interior) mutable; handle memberexpression lvalues
* lots of comments; hook return is frozen
* update main comment
* inference for switch, which reveals a bug
* fix yarn.lock
Please indicate if this issue affects the following tools provided by React Compiler.
options:
- label:React Compiler core (the JS output is incorrect, or your app works incorrectly after optimization)
- label:babel-plugin-react-compiler (build issue installing or using the Babel plugin)
- label:eslint-plugin-react-compiler (build issue installing or using the eslint plugin)
- label:react-compiler-healthcheck (build issue installing or using the healthcheck script)
- type:input
attributes:
label:Link to repro
description:|
Please provide a repro by either sharing a [Playground link](https://playground.react.dev), or a public GitHub repo so the React team can reproduce the error being reported. Please do not share localhost links!
placeholder:|
e.g. public GitHub repo, or Playground link
validations:
required:true
- type:textarea
attributes:
label:Repro steps
description:|
What were you doing when the bug happened? Detailed information helps maintainers reproduce and fix bugs.
Issues filed without repro steps will be closed.
placeholder:|
Example bug report:
1. Log in with username/password
2. Click "Messages" on the left menu
3. Open any message in the list
validations:
required:true
- type:dropdown
attributes:
label:How often does this bug happen?
description:|
Following the repro steps above, how easily are you able to reproduce this bug?
options:
- Every time
- Often
- Sometimes
- Only once
validations:
required:true
- type:input
attributes:
label:What version of React are you using?
description:|
Please provide your React version in the app where this issue occurred.
# Number of days of inactivity before a stale issue or PR is closed
days-before-close:7
# Number of issues or PRs to process per day
# API calls per run
operations-per-run:100
# --- Issues ---
@@ -43,4 +43,4 @@ jobs:
close-pr-message:>
Closing this pull request after a prolonged period of inactivity. If this issue is still present in the latest release, please ask for this pull request to be reopened. Thank you!
# PRs with these labels will never be considered stale
- Export `act` from `react` [f1338f](https://github.com/facebook/react/commit/f1338f8080abd1386454a10bbf93d67bfe37ce85)
## 18.3.0 (April 25, 2024)
This release is identical to 18.2 but adds warnings for deprecated APIs and other changes that are needed for React 19.
Read the [React 19 Upgrade Guide](https://react.dev/blog/2024/04/25/react-19-upgrade-guide) for more info.
### React
- Allow writing to `this.refs` to support string ref codemod [909071](https://github.com/facebook/react/commit/9090712fd3ca4e1099e1f92e67933c2cb4f32552)
- Warn for deprecated `findDOMNode` outside StrictMode [c3b283](https://github.com/facebook/react/commit/c3b283964108b0e8dbcf1f9eb2e7e67815e39dfb)
- Warn for deprecated `test-utils` methods [d4ea75](https://github.com/facebook/react/commit/d4ea75dc4258095593b6ac764289f42bddeb835c)
- Warn for deprecated Legacy Context outside StrictMode [415ee0](https://github.com/facebook/react/commit/415ee0e6ea0fe3e288e65868df2e3241143d5f7f)
- Warn for deprecated string refs outside StrictMode [#25383](https://github.com/facebook/react/pull/25383)
- Warn for deprecated `defaultProps` for function components [#25699](https://github.com/facebook/react/pull/25699)
- Warn when spreading `key` [#25697](https://github.com/facebook/react/pull/25697)
- Warn when using `act` from `test-utils` [d4ea75](https://github.com/facebook/react/commit/d4ea75dc4258095593b6ac764289f42bddeb835c)
### React DOM
- Warn for deprecated `unmountComponentAtNode` [8a015b](https://github.com/facebook/react/commit/8a015b68cc060079878e426610e64e86fb328f8d)
- Warn for deprecated `renderToStaticNodeStream` [#28874](https://github.com/facebook/react/pull/28874)
@@ -4,7 +4,7 @@ React is a JavaScript library for building user interfaces.
* **Declarative:** React makes it painless to create interactive UIs. Design simple views for each state in your application, and React will efficiently update and render just the right components when your data changes. Declarative views make your code more predictable, simpler to understand, and easier to debug.
* **Component-Based:** Build encapsulated components that manage their own state, then compose them to make complex UIs. Since component logic is written in JavaScript instead of templates, you can easily pass rich data through your app and keep the state out of the DOM.
* **Learn Once, Write Anywhere:** We don't make assumptions about the rest of your technology stack, so you can develop new features in React without rewriting existing code. React can also render on the server using Node and power mobile apps using [React Native](https://reactnative.dev/).
* **Learn Once, Write Anywhere:** We don't make assumptions about the rest of your technology stack, so you can develop new features in React without rewriting existing code. React can also render on the server using [Node](https://nodejs.org/en) and power mobile apps using [React Native](https://reactnative.dev/).
[Learn how to use React in your project](https://react.dev/learn).
@@ -69,7 +69,7 @@ Facebook has adopted a Code of Conduct that we expect project participants to ad
Read our [contributing guide](https://legacy.reactjs.org/docs/how-to-contribute.html) to learn about our development process, how to propose bugfixes and improvements, and how to build and test your changes to React.
### Good First Issues
### [Good First Issues](https://github.com/facebook/react/labels/good%20first%20issue)
To help you get your feet wet and get you familiar with our contribution process, we have a list of [good first issues](https://github.com/facebook/react/labels/good%20first%20issue) that contain bugs that have a relatively limited scope. This is a great place to get started.
React Compiler is a compiler that optimizes React applications, ensuring that only the minimal parts of components and hooks will re-render when state changes. The compiler also validates that components and hooks follow the Rules of React.
More information about the design and architecture of the compiler are covered in the [Design Goals](./docs/DESIGN_GOALS.md).
More information about developing the compiler itself is covered in the [Development Guide](./docs/DEVELOPMENT_GUIDE.md).
File diff suppressed because it is too large
Load Diff
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
