Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performant globbing and background analysis #569

Merged
merged 7 commits into from
Nov 28, 2022

Conversation

skovhus
Copy link
Collaborator

@skovhus skovhus commented Nov 27, 2022

When the server boots up we analyze all files matching a glob (default to **/*@(.sh|.inc|.bash|.command)) from the given rootPath. We do this in order to enable features across files (e.g. jump to definition). The current setup works okay for smaller workspaces, but doesn't perform well the rootPath contains too many files. Problems observed:

  • globbing (currently using node-glob) can take a very long time to complete!
  • we block the LSP server initialisation so the server doesn't respond to other requests
  • we can run out of memory and thereby crash the server

This should be fixed in this PR that:

  1. switches the globbing library to fast-glob which is a lot faster. Quick benchmarks shows the initial glob resolution to be 10X faster and I didn't find any issues compared to last time this was tried out ([WIP] More file extensions than .sh and shebang lookup #113)
  2. limit the amount of bash files we analyse in the background. This is configurable, but defaults to the first 5000 files matching the glob. The background analysis for 500 files takes around 3 seconds to analyze on my machine.
  3. we now initialize the server before starting the background parsing

The usefulness of the background analysis for 500 files might be questionable, but the performance is decent IMO. With Sourcing aware symbols completion and jump to definition this will become more interesting.

Resolves #301
Might solve #317, #316 #277

This ensures that the LSP server will respond to requests
while the background parsing is going on.
This should ensure that the LSP server does not crash and yield
a major performance increase for large repositories.

fast-glob returns the same results and is around 10x faster.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

VSCode: Repo with not really 21K shell scripts locks up language server in "Analyzing" loop
1 participant