-
Notifications
You must be signed in to change notification settings - Fork 10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix hang when parsing unclosed comment #37
Conversation
The CI failure is not related to this PR:
|
Thank you so much for diving into this! (my Windows debugging setup has been quite suboptimal - VNC on a EC2 instance ☠️ - #40 is going to need love soon)
Something in tests/test-syntax.cpp would be ace |
Ah, somehow it's not giving you access to the HF_TOKEN so no gated models for you :-S (did you try and run There also seems to be failures in test-capabilities, more worrying. Testing out your patch :-) |
The following doesn't seem to work (assuming i patched it right):
|
This is causing a parsing error on Windows that is not seen on Linux or macOS:
Also, the tests are failing on Windows. It seems like std::regex in the Microsoft STL may handle newlines differently. I think the reason |
Yes there's a lot going on on Windows (newlines being \r\n and MS's interpretations of stdc++ lib compliance being even more creative than Apple's). Hopefully #45 gets us going. |
@cebtenzzre thanks again for preparing this |
@ochafik I found a fix that works even with the nonconforming Microsoft STL, and added a test. #45 did not fix the issue. Could this PR be reopened? https://github.com/nomic-ai/minja/tree/fix-opencomment-hang |
Ohh, sorry about this / thanks for keeping at it! |
Doesn’t seem to let me reopen it, necromancy has limits it seems. Could you please open a new one? |
Done: #49 |
Since c73b53f, minja hangs when parsing the template
{#
, e.g.:This program should exit with status code 1, but instead it hangs.
I tried a different fix for the lack of
std::regex::multiline
on MSVC, and it no longer hangs. Also, all of the tests pass—it seems like the version withmultiline
was also buggy.multiline
makes anchors match at the end of a line, which I think is the opposite of what was intended.The ability to parse incomplete templates without crashing or hanging is important to GPT4All, which parses the template as it is being typed in the Settings page. This provides instant feedback on whether it is valid.
I'd like to add a regression test for this change. Let me know where it belongs. Or, you can add one if you'd like.