high CPU and memory usage on Linux

[LadySerena]

Description

I also encountered the same error as this issue on macos. I got about 10 seconds of perf events before I needed to exit the program, since it causes my system to become unresponsive. Let me know if you need more info. I'm still a newb with perf, so I wasn't able to sift out the signal from the noise.

Environment

uname output

Linux serena-desktop 6.1.1-arch1-1 #1 SMP PREEMPT_DYNAMIC Wed, 21 Dec 2022 22:27:55 +0000 x86_64 GNU/Linux  

helix version

helix 22.12  

bash-language-server version

bash-language-server --version
Version is 4.1.3  

node version running under nvm

nvm version
v18.12.1  

Related Issues

Looking at Helix's issue tracker there might be similar to this one but I'm not 100% sure.

Full Flamegraph

flamgraph

Steps to reproduce

Open my zshrc file

(On arch the packagers made the binary name helix instead of the default hx)

• hx $PATH_TO_FILE
• helix $PATH_TO_FILE

edit the helix-dev2 function and add a tmux command. This bug is that a bunch (20+) shellcheck processes spin up and exhaust the system of resources.

full zsh file

# If you come from bash you might have to change your $PATH.
# export PATH=$HOME/bin:/usr/local/bin:$PATH

# Path to your oh-my-zsh installation.
export ZSH="/home/serena/.oh-my-zsh"

# Set name of the theme to load --- if set to "random", it will
# load a random theme each time oh-my-zsh is loaded, in which case,
# to know which specific one was loaded, run: echo $RANDOM_THEME
# See https://github.com/ohmyzsh/ohmyzsh/wiki/Themes
ZSH_THEME="aphrodite"

# Set list of themes to pick from when loading at random
# Setting this variable when ZSH_THEME=random will cause zsh to load
# a theme from this variable instead of looking in $ZSH/themes/
# If set to an empty array, this variable will have no effect.
# ZSH_THEME_RANDOM_CANDIDATES=( "robbyrussell" "agnoster" )

# Uncomment the following line to use case-sensitive completion.
# CASE_SENSITIVE="true"

# Uncomment the following line to use hyphen-insensitive completion.
# Case-sensitive completion must be off. _ and - will be interchangeable.
# HYPHEN_INSENSITIVE="true"

# Uncomment the following line to disable bi-weekly auto-update checks.
# DISABLE_AUTO_UPDATE="true"

# Uncomment the following line to automatically update without prompting.
# DISABLE_UPDATE_PROMPT="true"

# Uncomment the following line to change how often to auto-update (in days).
# export UPDATE_ZSH_DAYS=13

# Uncomment the following line if pasting URLs and other text is messed up.
# DISABLE_MAGIC_FUNCTIONS=true

# Uncomment the following line to disable colors in ls.
# DISABLE_LS_COLORS="true"

# Uncomment the following line to disable auto-setting terminal title.
# DISABLE_AUTO_TITLE="true"

# Uncomment the following line to enable command auto-correction.
# ENABLE_CORRECTION="true"

# Uncomment the following line to display red dots whilst waiting for completion.
# COMPLETION_WAITING_DOTS="true"

# Uncomment the following line if you want to disable marking untracked files
# under VCS as dirty. This makes repository status check for large repositories
# much, much faster.
# DISABLE_UNTRACKED_FILES_DIRTY="true"

# Uncomment the following line if you want to change the command execution time
# stamp shown in the history command output.
# You can set one of the optional three formats:
# "mm/dd/yyyy"|"dd.mm.yyyy"|"yyyy-mm-dd"
# or set a custom format using the strftime function format specifications,
# see 'man strftime' for details.
# HIST_STAMPS="mm/dd/yyyy"

# Would you like to use another custom folder than $ZSH/custom?
# ZSH_CUSTOM=/path/to/new-custom-folder

# Which plugins would you like to load?
# Standard plugins can be found in $ZSH/plugins/
# Custom plugins may be added to $ZSH_CUSTOM/plugins/
# Example format: plugins=(rails git textmate ruby lighthouse)
# Add wisely, as too many plugins slow down shell startup.
# plugins=(git)
eval "$(hub alias -s)"
fpath+=~/.zfunc
fpath=(~/.zsh/completions $fpath)
autoload -U compinit && compinit
source $ZSH/oh-my-zsh.sh
export GITHUB_USER=LadySerena
export VAGRANT_DEFAULT_PROVIDER=libvirt
export GOPATH=$HOME/go
export BROWSER=firefox-developer-edition
export PATH="$GOPATH/bin:$PATH"
export PATH=$HOME/.rbenv/shims:$PATH
export PATH=$HOME/.local/bin:$PATH
source /usr/share/nvm/init-nvm.sh
# User configuration
source ~/.zsh_secrets
# export MANPATH="/usr/local/man:$MANPATH"

# k8s stuff

source $HOME/Code/jonmosco/kube-ps1/kube-ps1.sh
PROMPT='$(kube_ps1)'$PROMPT

# You may need to manually set your language environment
# export LANG=en_US.UTF-8

# Preferred editor for local and remote sessions
# if [[ -n $SSH_CONNECTION ]]; then
#   export EDITOR='vim'
# else
#   export EDITOR='mvim'
# fi

# Compilation flags
# export ARCHFLAGS="-arch x86_64"

# Set personal aliases, overriding those provided by oh-my-zsh libs,
# plugins, and themes. Aliases can be placed here, though oh-my-zsh
# users are encouraged to define aliases within the ZSH_CUSTOM folder.
# For a full list of active aliases, run `alias`.
#
# Example aliases
# alias zshconfig="mate ~/.zshrc"
# alias ohmyzsh="mate ~/.oh-my-zsh"
alias pr="git browse -- pulls"
alias pbcopy="xclip -selection c"
export PATH="$HOME/.jenv/bin:$PATH"
eval "$(jenv init -)"
eval "$(rbenv init - zsh)"

export EDITOR="helix"

export PATH="$(rbenv prefix)/bin:$PATH"

# The next line updates PATH for the Google Cloud SDK.
if [ -f '/home/serena/google-cloud-sdk/path.zsh.inc' ]; then . '/home/serena/google-cloud-sdk/path.zsh.inc'; fi

# The next line enables shell command completion for gcloud.
if [ -f '/home/serena/google-cloud-sdk/completion.zsh.inc' ]; then . '/home/serena/google-cloud-sdk/completion.zsh.inc'; fi

# functions that are helpful

rs3-snap() {
  snap run runescape.rs3
}

helix-dev2() {
  tmux new-window -c "$PWD"
  tmux split-window -v 'sidetree $PWD'
  tmux resize-pane
}

helix-dev() {
  tmux new-window -c "$PWD"
  tmux split-window -h "helix $PWD"
  tmux resize-pane -L 95
  tmux split-pane -v
  tmux send-keys -t 0 'sidetree' enter; 
  tmux resize-pane -D 20
  tmux select-pane -t 1
}

source /usr/share/fzf/key-bindings.zsh
source /usr/share/fzf/completion.zsh

[skovhus]

Thanks for the detailed error report.

Can you provide the flamegraph in another format that I can explore? And let me know how you created it, I’m curious if this was generated by Helix.

[skovhus]

This bug is that a bunch (20+) shellcheck processes spin up and exhaust the system of resources.

I overlooked this comment! That was also my hunch, so thanks for clarifying this. We should definitely throttle the shell check calls.

[LadySerena]

Thanks for the detailed error report.

Can you provide the flamegraph in another format that I can explore? And let me know how you created it, I’m curious if this was generated by Helix.

I generated the flamegraph following the instructions in this repo. (it would be interesting to have helix have a perf integration).

I pulled up my zsh history and got the commands I used. You use perf to launch the process and take samples at 100Hz and run for 10 seconds. You then parse the perf output and then use the perl scripts from the flamegraph repo to take that data and create the svg file.

There are other transformations one can do but I'm not the most adept at flamegraphs, and I included it since you asked for one in the MacOS issue I linked.

perf record -F 100 -g -- helix $SAMPLE_ZSHRC_FILE 10
perf script > out.perf
git clone https://github.com/brendangregg/FlameGraph.git
cd FlameGraph
./stackcollapse-perf.pl ../out.perf  > ../out.folded
./flamegraph.pl ../out.folded > ../flamegraph.svg

edited to add:

Can you provide the flamegraph in another format that I can explore?

Huh, today I learned one can upload a gzipped tar ball to a github issue comment. I have the raw perf output and the transformed files as well as the final flamegraph.

issue-data.tar.gz

[skovhus]

Thanks for the details!

A new version [email protected] has just been released, introducing a short execution delay and throttling for ShellCheck tasks. I'm still confused why ShellCheck can starve the system – I don't see that behaviour on OS X...

But I would like you to verify if the issue is resolved. Thanks. :)

[LadySerena]

Thanks for that fix. The throttling helped a lot. I took a small test run and opened my zshrc file and I didn't get a crash :D I did notice when I exited editing a shell file helix kept the language server open longer than needed, but that's helix's problem or my problem if it's a config issue.

[skovhus]

@LadySerena thanks. It is very odd that ShellCheck consumes so much CPU on Linux...

If you have time then I would appreciate if you tested out version 4.2.4 also that includes a simpler debouncing.

[LadySerena]

4.2.4 is way better! Still running a little toasty, but this is a much better state than when I initially opened this issue. Now, I only see the one shellcheck process spinning up

[skovhus]

Note that we have also fixed a bug causing memory leaks and high CPU usage for workspaces with many files – this has been released as [email protected] and the vscode extension 1.26.0.

[LadySerena]

Hey actually that last PR fixed the resource usage problem. Thank you for getting the patches together :D