Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement SynchronizationContext for GLib #856

Merged
merged 1 commit into from
May 14, 2023
Merged

Implement SynchronizationContext for GLib #856

merged 1 commit into from
May 14, 2023

Conversation

ultimaweapon
Copy link
Contributor

Closes #854

No unit tests yet. Need some feedback before adding a unit tests.

  • I agree that my contribution may be licensed either under MIT or any version of LGPL license.

@badcel
Copy link
Member

badcel commented Apr 20, 2023

I did not yet take a look at your code will do later.

To fix the build pipeline please run dotnet format GirCore.sln

Copy link
Member

@badcel badcel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your contribution. This looks promising. Please see my comments.

@badcel
Copy link
Member

badcel commented Apr 20, 2023

Please do not merge current main branch into this, but rebase your PR to avoid unnecessary commits.

@ultimaweapon
Copy link
Contributor Author

I'll comeback later to add unit tests so it is not ready for review yet.

@ultimaweapon ultimaweapon marked this pull request as ready for review April 26, 2023 03:10
@badcel
Copy link
Member

badcel commented Apr 26, 2023

Thanks for updating the PR. Please:

  • fix build errors
  • run dotnet format GirCore.sln to fix remaining formatting errors
  • squash your commits into one

I will do some in depth testing later.

@ultimaweapon
Copy link
Contributor Author

Strange. It was built successfully on my machine.

@ultimaweapon
Copy link
Contributor Author

I found the reason now. I did not run GenerateLibs.fsx after rebased. Now I have the same error.

@badcel
Copy link
Member

badcel commented Apr 27, 2023

As this is a fairly new topic for me please be patient with me, while I try to familiarize with the topic and your work, as I'm going to ask questions and having comments.

  • In general I think there is a unit tests missing for the case an exception is thrown.
  • Do you know how to test the Send method? The current test only covers Post.

I'm not sure if the unit test you wrote is correct. I did not take a look into the internals but just thought that your implementation fixes the use case you described in your unit test. So I went ahead and removed the line SynchronizationContext.SetSynchronizationContext(new Internal.GLibSynchronizationContext()); from Module.Initialize expecting your test to fail. Actually it did not fail on my system (Linux, fedora 38). Is the test failing for you if you disable the new SynchronizationContext? Which operating System are you using?

In regard to code style I would expect at least a comment if you call an async method without awaiting it. Additionally in C# there is a coding convention to suffix async methods with the word Async. So perhaps renaming AsyncMethod to DelayActionAsync (as it describes its behavior) is more appropriate? It would be nice if you could add a comment why you do not await the async method just before the method call.

@ultimaweapon
Copy link
Contributor Author

Do you know how to test the Send method? The current test only covers Post.

I also don't have any idea to test this without invoking it directly. Let me know if you want this test.

In general I think there is a unit tests missing for the case an exception is thrown.

Same answer as the above.

So I went ahead and removed the line SynchronizationContext.SetSynchronizationContext(new Internal.GLibSynchronizationContext()); from Module.Initialize expecting your test to fail. Actually it did not fail on my system (Linux, fedora 38). Is the test failing for you if you disable the new SynchronizationContext? Which operating System are you using?

I'm using Arch Linux. Here is what happens on my machine when I removed the GLibSynchronizationContext:

----- Test Execution Summary -----

GLib.Tests.SynchronizationContextTest.WithGLibSynchronizationContextAsyncShouldResumeOnMainLoop:
    Outcome: Failed
    Error Message:
    Expected resumeThread to be 4, but found 7.
    Stack Trace:
       at FluentAssertions.Execution.LateBoundTestFramework.Throw(String message)
   at FluentAssertions.Execution.TestFrameworkProvider.Throw(String message)
   at FluentAssertions.Execution.DefaultAssertionStrategy.HandleFailure(String message)
   at FluentAssertions.Execution.AssertionScope.FailWith(Func`1 failReasonFunc)
   at FluentAssertions.Execution.AssertionScope.FailWith(Func`1 failReasonFunc)
   at FluentAssertions.Execution.AssertionScope.FailWith(String message, Object[] args)
   at FluentAssertions.Numeric.NumericAssertions`2.Be(T expected, String because, Object[] becauseArgs)
   at GLib.Tests.SynchronizationContextTest.WithGLibSynchronizationContextAsyncShouldResumeOnMainLoop() in /home/ultimaweapon/Workspace/personal/gir.core/src/Tests/Libs/GLib-2.0.Tests/SynchronizationContextTest.cs:line 26
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)

    
Total tests: 1. Passed: 0. Failed: 1. Skipped: 0

@cameronwhite
Copy link
Contributor

It also fails reliably on my machine if I remove the synchronization context, though I don't know if it's guaranteed to always run on a different thread without it or if it might still be allowed to run on the main thread.

@badcel
Copy link
Member

badcel commented Apr 29, 2023

I think last time I screwed somehow up. Today on my system disabling the GLibSynchronizationContext failed the test, too. I think the code did not get recompiled properly on my system after disabling the class.

Can you add the inverse test, too? This would be the code:

    [TestMethod]
    public void WithoutGLibSynchronizationContextAsyncShouldNotResumeOnMainLoop()
    {
        SynchronizationContext.SetSynchronizationContext(new SynchronizationContext());

        var mainLoop = new MainLoop();
        int? resumeThread = null;

        AsyncMethod(() =>
        {
            resumeThread = System.Threading.Thread.CurrentThread.ManagedThreadId;
            mainLoop.Quit();
        });

        mainLoop.Run();

        resumeThread.Should().NotBe(System.Threading.Thread.CurrentThread.ManagedThreadId);
    }

I wanted to go a bit into the topic of exceptions to see if we need to implement something similar to Application.DispatcherUnhandledException. I tried to throw an exception like this:

        AsyncMethod(() =>
        {
            throw new Exception();
        });

But I was not able to catch it. You mentioned to "swallow" the Exception. If this could be done it should be possible to avoid crashing the runtime. If an exception occurs we could let the user gracefully handle the situation. How would you catch those exceptions?

@ultimaweapon
Copy link
Contributor Author

Yeah we need something like Application.DispatcherUnhandledException to handle the exception, which I think it is out of scope of this PR. I think it should be fixed by #845. Let me know what do you think.

For the inverse test there is a problem because SynchronizationContext is operate at the global level. That mean we need to disable parallel test execution on GLib-2.0.Tests otherwise it is possible for race conditions. IMO it is not worth because it will slow down a lot in the future when more tests are available and we already have a test that make sure the code is working as expected. Let me know if you want to proceed with the inverse test.

@badcel
Copy link
Member

badcel commented Apr 30, 2023

Yeah you are probably right we don't need to test things which are not working. It somehow defeats the purpose of tests.

In regard to exception handling I agree that there needs to be another PR but probably not for #845 but for some infrastructure classes which allow to signal an unhandled exception. I would like to land this PR first. Then your PR can use this code already and #845 can use it, too.

To be able to test this it would be nice if you could add a comment with the solution to catch an exception. If you want to, you can create this PR, too. Otherwise I would do it.

Regarding your statement that this operates on a "global" level it got me thinking. If there is no mainloop the new SynchronizationContext would probably not work?

So I wonder if the new SynchronizationContext should have a static method like "Ensure()" which sets it as the context for the current thread.

This method should probably not be called in "Module.Initialize" but perhaps in the static constructor of "Gio.Application" and "Glib.MainLoop". To enable it only if the user does something what actually runs a "MainLoop". Perhaps the code must even be run in the regular constructor as there could be different main loops running at the same time.

Then we would probably need another test that checks that async calls work even if no explicit SynchronizationContext is set.

If the MainLoop call is not called Automatically for "GTK.Application" or "Adw.Application" it needs to be added there, too. What do you think?

@ultimaweapon
Copy link
Contributor Author

In regard to exception handling I agree that there needs to be another PR but probably not for #845 but for some infrastructure classes which allow to signal an unhandled exception. I would like to land this PR first. Then your PR can use this code already and #845 can use it, too.

Okay sounds good. Let me know when it is available so I can update my PR.

To be able to test this it would be nice if you could add a comment with the solution to catch an exception. If you want to, you can create this PR, too. Otherwise I would do it.

I think I don't have enough capacity to work on this PR. AFAIK the only way to prevent the exceptions from being crossing the managed boundary is wrapping everything in the callback with try/catch something like this:

private static void ScheduleAction(Action action)
{
    var proxy = new SourceFuncNotifiedHandler(() =>
    {
        try
        {
            action();
        }
        catch (Exception ex)
        {
            // Do something with the exception.
        }

        return false;
    });

    Internal.Functions.IdleAdd(GLib.Constants.PRIORITY_DEFAULT_IDLE, proxy.NativeCallback, IntPtr.Zero, proxy.DestroyNotify);
}

Regarding your statement that this operates on a "global" level it got me thinking. If there is no mainloop the new SynchronizationContext would probably not work?

Right because the callback will never be executed. I was wrong about the race condition on SynchronizationContext because it is per-thread but it seems like you already aware this.

I agree that we should change the way to install the SynchronizationContext. According to this on Stack Overflow:

In general, you shouldn't install a synchronization context on a thread you don't own. If you do place one on a thread pool thread, it should be removed before the thread is returned to the thread pool. More likely, if you're installing a synchronization context, you just shouldn't ever give the thread back (a custom synchronization context is commonly associated with a "main loop" for that thread).

I think the best place to install it is just before the call to g_main_loop_run and g_application_run then remove it when those functions is returned.

@badcel
Copy link
Member

badcel commented Apr 30, 2023

Yeah this sounds best. But it will be hard to implement. We need to check or create some extension points somehow.

I already tried the try / catch block but for some reason it did not work. I hope it was not my ide failing me again. @cameronwhite if you have some capacity could you add some try / catch blocks down the call hierarchy and confirm that an exception is not caught?

I really would like to land this as it seems to be some pretty import building block I think.

@ultimaweapon
Copy link
Contributor Author

I just tested and found out AppDomain.UnhandledException has been raised before the crash, which mean the exception is not actually cross the managed boundary so this does not trigger the undefined behavior on the C side. The problem with AppDomain.UnhandledException is once it is triggered the application will be terminated so if we want some mechanism to report the exception and continue the execution it is not a viable choice.

I already tried the try / catch block but for some reason it did not work.

I updated ScheduleAction to be the same as my above comment and it is working as expected on my machine.

Yeah this sounds best. But it will be hard to implement. We need to check or create some extension points somehow.

Can we do it in the Run method on src/Libs/GLib-2.0/Public/MainLoop.cs something like this?

public void Run()
{
    var previousContext = SynchronizationContext.Current;

    SynchronizationContext.SetSynchronizationContext(new GLibSynchronizationContext());

    try
    {
        Internal.MainLoop.Run(Handle);
    }
    finally
    {
        SynchronizationContext.SetSynchronizationContext(previousContext);
    }
}

And did something similar to src/Libs/Gio-2.0/Public/Application.cs.

@badcel
Copy link
Member

badcel commented Apr 30, 2023

I updated ScheduleAction to be the same as my above comment and it is working as expected on my machine.

Thanks for testing and sharing. I will evaluate again. Perhaps there is some bug regarding compilation in Rider.

Can we do it in the Run method on src/Libs/GLib-2.0/Public/MainLoop.cs something like this?

The problem is that the code is autogenerated and the generator does not know that special code should be executed. An easy solution would be to have a method "RunWithSynchronizationContext" and keep the original method alone.

@badcel
Copy link
Member

badcel commented May 3, 2023

I tested again as we got support for UnhandledExceptions with #861.

Turns out that putting the try / catch block in the position works. The problem was in my unit test. As i did not quit the main loop in case of an exception the test was running indefinitely.

I updated the Run method to look like:

var previousContext = SynchronizationContext.Current;
SynchronizationContext.SetSynchronizationContext(new MainLoopSynchronizationContext());

try
{
    Internal.MainLoop.Run(Handle);
}
finally
{
    SynchronizationContext.SetSynchronizationContext(previousContext);
}

I updated the MainLoopSynchronizationContext.ScheduleAction to:

private static void ScheduleAction(Action action)
{
    var proxy = new SourceFuncNotifiedHandler(() =>
    {
        try
        {
            action();
        }
        catch(Exception ex)
        {
            UnhandledException.Raise(ex);
        }
        return false;
    });

    Internal.Functions.IdleAdd(GLib.Constants.PRIORITY_DEFAULT_IDLE, proxy.NativeCallback, IntPtr.Zero, proxy.DestroyNotify);
}

This test is green for me (I renamed GLibSynchronizationContext to MainLoopSynchronizationContext):

[TestMethod]
public void Test()
{
    var exceptionCaught = false;
    SynchronizationContext.SetSynchronizationContext(new MainLoopSynchronizationContext());
    var mainLoop = new MainLoop();
       
    GLib.UnhandledException.SetHandler(e =>
    {
        exceptionCaught = true;
        mainLoop.Quit();
    });

    AsyncMethod(() =>
    {
        throw new Exception();
    });

    mainLoop.Run();

    exceptionCaught.Should().BeTrue();
 }

There is one issue: As a workaround I set the synchronization context at the beginning. I tried to set the synchronization context only in Run as suggested by @ultimaweapon. But it looks like the AsyncMethod is not using the new MainLoopSynchronizationContext as the hidden async statemachine starts working immediately, meaning befor the call to Run thus missing out on the new MainLoopSynchronizationContext which will be set then.

From my point of view this is only a problem of the Unit-Test. Any ideas how to circumvent this problem?

If this is fixed I think this PR could be merged if all the discussed changes are applied.

@badcel
Copy link
Member

badcel commented May 4, 2023

@ultimaweapon I think I worked it out. To get your PR merged please do the following:

  • If Manual GLib implementations #863 is merged, please rebase your code on main as it contains some manual bindings which are needed for your PR.
  • Please do not add additional commits to your PR, but amend any changes to your current commit.
  • Please revert your changes in Module.cs
  • Please rename GLibSynchronizationContext to MainLoopSynchronizationContext
  • Please throw a NotImplementedException in MainLoopSynchronizationContext.Send and remove any other code in this metod as we currently don't know how to test it.
  • Please add a new method (in addition to the existing Run method)RunWithSynchronizationContext in MainLoop.cs and Gio.Application.cs like you suggested here
  • Please add exception handling to the MainLoopSynchronizationContext:
private static void ScheduleAction(Action action)
{
    var proxy = new SourceFuncNotifiedHandler(() =>
    {
        try
        {
            action();
        }
        catch(Exception ex)
        {
            UnhandledException.Raise(ex);
        }
        return false;
    });

    Functions.IdleAdd(Constants.PRIORITY_DEFAULT_IDLE, proxy.NativeCallback, IntPtr.Zero, proxy.DestroyNotify);
}
  • Please use the following code to test your canges:
using System;
using System.Threading.Tasks;
using FluentAssertions;
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace GLib.Tests;

[TestClass, TestCategory("IntegrationTest")]
public class SynchronizationContextTest : Test
{
    [TestMethod]
    public void AsyncMethodIsExecutedOnMainLoopThread()
    {
        var mainLoop = new MainLoop();
        var context = mainLoop.GetContext();
        var source = Functions.TimeoutSourceNew(1);
        source.Attach(context);

        int? resumeThread = null;
        source.SetCallback(() =>
        {
            AsyncMethod(() =>
            {
                resumeThread = System.Threading.Thread.CurrentThread.ManagedThreadId;
                mainLoop.Quit();
            });
            return false;
        });

        mainLoop.RunWithSynchronizationContext();

        resumeThread.Should().Be(System.Threading.Thread.CurrentThread.ManagedThreadId);
    }

    [TestMethod]
    public void ExceptionInAsyncMethodCanBeHandledViaUnhandledExceptionHandler()
    {
        var mainLoop = new MainLoop();
        var context = mainLoop.GetContext();
        var source = Functions.TimeoutSourceNew(1);
        source.Attach(context);
        source.SetCallback(() =>
        {
            AsyncMethod(() =>
            {
                throw new Exception();
            });
            return false;
        });

        var exceptionCaught = false;
        UnhandledException.SetHandler(e =>
        {
            exceptionCaught = true;
            mainLoop.Quit();
        });

        mainLoop.RunWithSynchronizationContext();

        exceptionCaught.Should().BeTrue();
    }
    
    private static async void AsyncMethod(Action finish)
    {
        await Task.Delay(1);
        finish();
    }
}

@ultimaweapon
Copy link
Contributor Author

Sorry for late response. I'm having some personal problems so I can't working on this until the end of next week. If you are in a hurry you can take over this PR.

@badcel badcel added this to the 0.4.0 milestone May 7, 2023
@ultimaweapon
Copy link
Contributor Author

Done. Let me know if there is any problem.

Copy link
Member

@badcel badcel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good work and thank you for your patience with me. In case you are missing more features more contributions are very welcome.

@badcel badcel merged commit 77a86c2 into gircore:main May 14, 2023
@ultimaweapon ultimaweapon deleted the sync-context branch May 14, 2023 18:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support synchronization context to avoid manual code dispatch
3 participants