tests: enable core24 plugin tests #5209

sergiusens · 2025-01-17T15:45:04Z

Have you followed the guidelines for contributing?
Have you signed the CLA?
Have you successfully run tox run -m lint?
Have you successfully run tox run -e test-py310? (supported versions: py39, py310, py311, py312)

Signed-off-by: Sergio Schvezov <[email protected]>

Issue raised: #5207 Signed-off-by: Sergio Schvezov <[email protected]>

- Remove the dotnet test for core24 from build and run - Move the dotnet plugin test to the plugin suite Signed-off-by: Sergio Schvezov <[email protected]>

Signed-off-by: Sergio Schvezov <[email protected]>

bepri

That's a lot of movin' around! 20.04 spread test failures seem to be happening on other branches, so likely unrelated.

sergiusens · 2025-01-17T20:57:04Z

@bepri I still need to move those other python tests!

tigarmo · 2025-01-17T21:03:11Z

spread.yaml

+ tests/spread/core24-suites/plugins/:
+   summary: core24 plugin tests
+   systems:
+     - ubuntu-24.04*


do these tests build destructively?
One argument "pro" using something lower than 24.04 is that it lets us catch failures in classic snaps

tigarmo · 2025-01-17T21:06:19Z

tests/spread/core24-suites/plugins/colcon-msg-package/task.yaml

+  SNAP/colcon_msg_package: colcon-msg-package
+
+systems:
+  - ubuntu-22.04


these are using 22.04, is that on purpose?

no, but I will drop it from here and make the suite use 22.04

fwiw, they could be handled by the global suite systems filter, which I switched to 22.04, but these tests will not work on non amd64 or arm64, so the filter is more specific

Signed-off-by: Sergio Schvezov <[email protected]>

…read' into work/CRAFT-3138-core24-plugin-spread

These tests were broken because the prime directory wasn't being made in managed builds

…ld in managed mode

tigarmo

thanks! One thing that looks "unclean" on these tests in general is the existence of spread/plugins/v2 referring to core20, spread/plugins/craft-parts referring to core22 and spread/core24-suites/plugins for core24. Seems a bit too cryptic but I don't think we need to address that in this PR

tigarmo · 2025-02-25T14:40:57Z

tests/spread/core24-suites/environment/test-variables/task.yaml

-  #shellcheck source=tests/spread/tools/snapcraft-yaml.sh
-  . "$TOOLS_DIR/snapcraft-yaml.sh"
-  # set_base "../snaps/$SNAP/snap/snapcraft.yaml"
+  SNAPCRAFT_BUILD_ENVIRONMENT: host


How come this test now runs on host?

It always must've. The content of the test itself requires destructive mode in order for the text files in all of the override-* steps to be created. I'm guessing it being destructive was set by the suite settings or something previously

tigarmo · 2025-02-25T14:44:26Z

tests/spread/core24-suites/manifest/manifest-info-envvars/task.yaml

@@ -1,5 +1,8 @@
 summary: Test manifest file creation

+systems:
+  - ubuntu-24.04*


This test runs snapcraft --use-lxd, so presumably the host doesn't need to be noble. Is there a specific reason for this?

It also runs --destructive-mode a couple lines later, so it needed to be noble to build core24 snaps

tigarmo · 2025-02-25T14:47:08Z

tests/spread/core24-suites/plugins/build-and-run-hello/poetry-hello/snap/snapcraft.yaml

+summary: simple python application
+description: build a python application using core24
+base: core24
+confinement: strict


for the future we should also have classic versions of this test, for poetry, python and uv

tigarmo · 2025-02-25T14:52:39Z

tests/spread/core24-suites/plugins/python-hello/classic/snap/snapcraft.yaml

this set of tests feels a bit redundant with those in build-and-run-hello now; should we address it in this PR?

I think this is a super noisy PR as-is and has been up for a while, but I'm happy to talk about doing this in another PR

tigarmo · 2025-02-25T14:57:35Z

.github/workflows/spread-scheduled.yaml

@@ -78,3 +78,51 @@ jobs:
            google:ubuntu-22.04-64:tests/spread/core22/remote-build \
            google:ubuntu-24.04-64:tests/spread/core24/remote-build \
            google:fedora-39-64:tests/spread/core24/remote-build:no_platforms
+            google:ubuntu-22.04-64:tests/spread/core24-suites/remote-build


what's the difference between tests/spread/core24 and tests/spread/core24-suites?

core24 is a suite in itself, we could play a dance and move everything from core24 into core24/general and then move everything from core24-suites into core24.

We seen plenty of times people adding spread tests that look good but never get run because it was a suite introduced in a suite.

we could play a dance and move everything from core24 into core24/general and then move everything from core24-suites into core24.

This seems reasonable, at least it would be less confusing. Looking at this from another perspective: how should one decide where to put a new test?
but I'm fine either way I suppose.

Does tests/spread/core24-suites/remote-build exist? I just kicked off a scheduled spread test to see what runs.

https://github.com/canonical/snapcraft/actions/runs/13531381365

It does not! Thanks for catching.

if it is base specific, and potentially for multiple bases, I believe repeating here is worth it, we have not had more than 3 bases active at any given time

mr-cal

Everything looks good but I'd like to see the scheduled spread test complete before landing

mr-cal · 2025-02-25T21:35:04Z

.github/workflows/spread-scheduled.yaml

@@ -78,3 +78,51 @@ jobs:
            google:ubuntu-22.04-64:tests/spread/core22/remote-build \
            google:ubuntu-24.04-64:tests/spread/core24/remote-build \
            google:fedora-39-64:tests/spread/core24/remote-build:no_platforms
+            google:ubuntu-22.04-64:tests/spread/core24-suites/remote-build


Does tests/spread/core24-suites/remote-build exist? I just kicked off a scheduled spread test to see what runs.

https://github.com/canonical/snapcraft/actions/runs/13531381365

The kernel plugin is only available as an experimental plugin, which isn't supported by snapcraft in core24, so this test isn't possible to run.

sergiusens added 7 commits January 17, 2025 07:04

tests: create the core24 plugins suite

fdb57a5

Signed-off-by: Sergio Schvezov <[email protected]>

tests: set base to core24 for the plugin suite

cde70e4

Signed-off-by: Sergio Schvezov <[email protected]>

tests: update python interpreter for core24

2dfb62a

Signed-off-by: Sergio Schvezov <[email protected]>

tests: update ros2 plugin tests for core24

2f90b1a

Signed-off-by: Sergio Schvezov <[email protected]>

tests: disable python core24 and gnome extension

cc26e17

Issue raised: #5207 Signed-off-by: Sergio Schvezov <[email protected]>

tests: reorganize dotnet

edf6221

- Remove the dotnet test for core24 from build and run - Move the dotnet plugin test to the plugin suite Signed-off-by: Sergio Schvezov <[email protected]>

tests: remove base setting from core24 plugin test

1e9facc

Signed-off-by: Sergio Schvezov <[email protected]>

sergiusens requested review from tigarmo and bepri January 17, 2025 15:45

bepri approved these changes Jan 17, 2025

View reviewed changes

bepri self-requested a review January 17, 2025 21:00

tigarmo reviewed Jan 17, 2025

View reviewed changes

sergiusens and others added 11 commits January 20, 2025 17:00

tests: switch core24 plugin suite to run on 22.04

f73ff54

Signed-off-by: Sergio Schvezov <[email protected]>

chore: update craft parts to 2.3.0

53867c8

Signed-off-by: Sergio Schvezov <[email protected]>

chore: merge main

2fee9af

chore: migrate core24 plugin tests to the suite

0e955f9

Signed-off-by: Sergio Schvezov <[email protected]>

chore: migrate npm to build and run for core24

ae2a07a

Signed-off-by: Sergio Schvezov <[email protected]>

tests: run core24 plugin steps individually

967d57f

Signed-off-by: Sergio Schvezov <[email protected]>

Merge branch 'main' into work/CRAFT-3138-core24-plugin-spread

e85929a

tests: add uv tests to core24 plugin suite

d4b3da7

tests: use core24 for core24 tests

49c5533

test: don't use destructive builds for python tests

22e7808

Merge branch 'main' into work/CRAFT-3138-core24-plugin-spread

3335fc0

bepri requested review from tigarmo and removed request for bepri February 13, 2025 14:20

bepri and others added 4 commits February 13, 2025 13:14

test: make python plugin tests run in managed mode

826973f

Merge remote-tracking branch 'origin/work/CRAFT-3138-core24-plugin-sp…

7ac2f05

…read' into work/CRAFT-3138-core24-plugin-spread

Merge branch 'main' into work/CRAFT-3138-core24-plugin-spread

3bf9342

test: fix python symlink spread tests

2e04c38

These tests were broken because the prime directory wasn't being made in managed builds

bepri linked an issue Feb 18, 2025 that may be closed by this pull request

Migrate or copy core20 and core22 spread tests to core24 #4932

Open

bepri added 3 commits February 18, 2025 16:23

test: pr feedback, improve spread restore

bbcb742

ci: update scheduled workflow with new manual tests

ea27381

test(docs): clarify why the python tests are run in managed mode

858f63e

mr-cal mentioned this pull request Feb 20, 2025

test: use lxd for python symlink tests #5273

Open

4 tasks

bepri added 5 commits February 20, 2025 14:11

test: make all core24 suites run in managed mode on core22

9a6dadb

test: avoid mangling the project files before spread runs, always bui…

7323f44

…ld in managed mode

test: forgot to save a file

bfc9252

test: actually use managed mode, fix some syntax errors in spread.yaml

258e4ed

test: pin platform-dependent/destructive tests to core24

0c323b0

bepri force-pushed the work/CRAFT-3138-core24-plugin-spread branch from 85292f9 to 0c323b0 Compare February 21, 2025 14:42

bepri added 9 commits February 21, 2025 09:43

Merge branch 'main' into work/CRAFT-3138-core24-plugin-spread

ac30910

test: set variables test to run in managed mode

2ea0504

test: adjust empty scriptlet test for managed mode

d1bb235

test: update colcon tests to use ros2-jazzy

fdc11fe

test: adjust scriptlet tests to succeed in destructive mode for speed

21dd9ba

test: check for jazzy packages in ros2 tests

20338a7

test: run colcon tests in destructive mode

a50201f

test: move colcon tests to slow manual scheduled tests

3e764ce

test: make python symlink tests run in destructive mode for speed

7d588bc

bepri added the squash label Feb 24, 2025

tigarmo reviewed Feb 25, 2025

View reviewed changes

tigarmo approved these changes Feb 25, 2025

View reviewed changes

mr-cal reviewed Feb 25, 2025

View reviewed changes

lengau self-requested a review February 26, 2025 01:32

lengau approved these changes Feb 26, 2025

View reviewed changes

bepri added 4 commits February 26, 2025 08:46

test: remove reference to nonexistent test

440f2d5

test: remove kernel test

42e586b

The kernel plugin is only available as an experimental plugin, which isn't supported by snapcraft in core24, so this test isn't possible to run.

test: update matter-sdk version in tests

02e5db2

test: fix task for matter-sdk test

ba41156

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

tests: enable core24 plugin tests #5209

tests: enable core24 plugin tests #5209

sergiusens commented Jan 17, 2025

bepri left a comment •

edited

Loading

sergiusens commented Jan 17, 2025

tigarmo Jan 17, 2025

tigarmo Jan 17, 2025

sergiusens Jan 17, 2025

sergiusens Jan 20, 2025

tigarmo left a comment

tigarmo Feb 25, 2025

bepri Feb 25, 2025

tigarmo Feb 25, 2025

bepri Feb 25, 2025

tigarmo Feb 25, 2025

tigarmo Feb 25, 2025

bepri Feb 25, 2025

tigarmo Feb 25, 2025

sergiusens Feb 25, 2025

tigarmo Feb 25, 2025

mr-cal Feb 25, 2025

bepri Feb 26, 2025

sergiusens Feb 26, 2025

mr-cal left a comment

mr-cal Feb 25, 2025

tests: enable core24 plugin tests #5209

Are you sure you want to change the base?

tests: enable core24 plugin tests #5209

Conversation

sergiusens commented Jan 17, 2025

bepri left a comment • edited Loading

Choose a reason for hiding this comment

sergiusens commented Jan 17, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tigarmo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mr-cal left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bepri left a comment •

edited

Loading