teuth-suite: Add dry-run for scheduling suites #3
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Vallari Agrawal <[email protected]>
- Add two query params (dry_run and logs) to /suite - Use multiprocessing to isolate logs of two parallel runs Signed-off-by: Vallari Agrawal <[email protected]>
Signed-off-by: Vallari Agrawal <[email protected]>
Signed-off-by: Vallari Agrawal <[email protected]>
zmc
approved these changes
Sep 14, 2022
There was a problem hiding this comment.
Clever idea to use multiprocessing to redirect the logging! A couple feedback items that I don't consider a blocker for this PR:
- The logfile path should probably be configurable, but adding a configuration system should probably be its own PR
create_runwill take a significant amount of time to execute; likely longer than a client's default timeout for http requests. We may need to place these requests into a queue and return an ID, so the client can check the status of a scheduling request later on. celery might be a good choice here. This is a larger topic, but just wanted to mention it before I forgot 😄
Signed-off-by: Vallari Agrawal <[email protected]>
|
@kamoltat should we merge this PR? I think this was approved over call. And the |
kamoltat
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add two query params
dry_run(bool) andlogs(bool).If
dry_runis true, don't schedule suite and return logs (regardless of the value oflogsparam).If
logsis true, return logs (for both normal and dry-run).After scheduling the suite, query paddles for run details.
If run is not found, throw error.
To separate logs of two parallel runs, multiprocessing is used.
Why multiprocessing?
For two parallel runs, teuthology was printing logs to the same file, and configuring those runs to print to different files was not working as expected. It was not able to identify difference between the source of logs from those two parallel runs.
So, I wanted to get a new instance of the teuthology import, for which multiprocessing seemed like a good solution.
Requested changes