You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This could arguably be user (spec writer?) error, but...
I just discovered <wpt>, and it's a really neat feature that I'm excited to integrate into Secure Payment Confirmation's spec. However, I discovered that the test 'summary' view does not seem to deduplicate test names for its summary numbers, and so effectively restricts one from naming the same test twice in a spec.
(Example from SPC - we only have 48 tests in total, but when I linked the same test file from multiple places it counted each instance separately to get 71 tests instead! And I have no idea what happened to Firefox or Safari...).
A reasonable answer might be 'split up your test files more', but although we are definitely guilty of too much behavior tested in one test, I think it is reasonable for a test file to test more than a single location in a specification.
Perhaps the test summary component could throw the test file names into a set() before computing the summary?
The text was updated successfully, but these errors were encountered:
Being able to list a test twice (to put a cross-cutting test next to both things it exercises, for example) is absolutely a supported use case, so this is a bug on my part.
Previously, every .wpt-name was counted for calculating the pass/total summary,
even if it had previously been seen. This would cause inaccurate results for
specifications where authors list the same WPT test more than once (e.g., if
the test relates to multiple parts of the spec).
Fixesspeced#2303
Being able to list a test twice (to put a cross-cutting test next to both things it exercises, for example) is absolutely a supported use case, so this is a bug on my part.
Good to hear :). I've sent a PR for this that I hope is correct, so please (when you have time, no rush!) take a look and lmk. I think it does need your permission to run the checks as I am a 'first time contributor' here :D.
This could arguably be user (spec writer?) error, but...
I just discovered
<wpt>
, and it's a really neat feature that I'm excited to integrate into Secure Payment Confirmation's spec. However, I discovered that the test 'summary' view does not seem to deduplicate test names for its summary numbers, and so effectively restricts one from naming the same test twice in a spec.
(Example from SPC - we only have 48 tests in total, but when I linked the same test file from multiple places it counted each instance separately to get 71 tests instead! And I have no idea what happened to Firefox or Safari...).A reasonable answer might be 'split up your test files more', but although we are definitely guilty of too much behavior tested in one test, I think it is reasonable for a test file to test more than a single location in a specification.
Perhaps the test summary component could throw the test file names into a
set()
before computing the summary?The text was updated successfully, but these errors were encountered: