internal: Load VFS config changes in parallel #17771
Conversation
rustbot
added
the
S-waiting-on-review
|
@davidbarsky you most likely want to try this with your stuff before we think about merging this |
Veykril
force-pushed
the
parallel-vfs-config
branch
from
1e3686a to
8286847
Compare
|
Thanks! I assumed it wasn't possible to improve the VFS loading side of things without a wholesale replacement, but testing this against Buck2 on an Eden-backed file system on macOS (where rust-analyzer would be loading 909 crates) roughly halved the initial 1 "Root Scanned" duration from 46 seconds to roughly 20 seconds. There were cases where the "Roots Scanned" duration was as fast as 6 seconds (!) with these changes, but I wasn't able to consistently reproduce those figures. I'll try this on my remote Linux VM now! Footnotes
|
|
It's not a perfect solution as we will still re-switch multiple times in certain scenarios (as in we still dont wait for the initial loadup to finish), but it should reduce the number of times we restart the switching due to this speeding up the initial VFS load up. |
|
Though somethings wrong with the logic it seems given the failing test |
Oh, yeah: don't get me wrong: I'll happily take an incremental win over a wholesale replacement; I didn't realize it was possible to have an incremental win here! (I tried a similar approach to this PR and abandoned it when I didn't see a performance difference). Reducing the cost of switching workspaces sounds like a win in my book :) |
|
I imagine the "waiting for VFS loading finished before switching workspaces" should also not be too difficult, as its basically just delaying switching until a flag is set. Its mainly annoying to do with the opqueue setup I think. |
| !(self.last_reported_status.is_none() | ||
| || self.fetch_workspaces_queue.op_in_progress() | ||
| || self.fetch_build_data_queue.op_in_progress() | ||
| || self.fetch_proc_macros_queue.op_in_progress() | ||
| || self.discover_workspace_queue.op_in_progress() | ||
| || self.vfs_progress_config_version < self.vfs_config_version | ||
| || self.vfs_progress_n_done < self.vfs_progress_n_total) | ||
| self.vfs_done | ||
| && self.last_reported_status.is_some() | ||
| && !self.fetch_workspaces_queue.op_in_progress() | ||
| && !self.fetch_build_data_queue.op_in_progress() | ||
| && !self.fetch_proc_macros_queue.op_in_progress() | ||
| && !self.discover_workspace_queue.op_in_progress() | ||
| && self.vfs_progress_config_version >= self.vfs_config_version |
There was a problem hiding this comment.
I flipped this because I personally have difficult reading the previous negation
|
@bors r+ |
|
☀️ Test successful - checks-actions |
Summary: The rust-analyzer dependency is currently 2024-07-29. It turns out that a few days after that an important performance improvement landed, which does parallel loading for file in VFS. This has a significant impact on initial startup time. This diff does the bare minimum to move to the first subsequent release that had this feature, from [2024-08-12](https://github.com/rust-lang/rust-analyzer/releases/tag/2024-08-12). The key change is [this one](rust-lang/rust-analyzer#17771) "load VFS config changes in parallel." Reviewed By: jcpetruzza, michalmuskala Differential Revision: D72724277 fbshipit-source-id: eac8f371acfa548df0c51fcf2c6c49fe5417f36f
Simple attempt to make some progress f or #17373
No clue if those atomic orderings are right, though I don't think they are really too relevant either.
A more complete fix would probably need to replace our
ProjectFoldershandling a bit.