tools: migrate test generation features to `gopls` #1594

emil14 · 2021-06-28T10:24:56Z

Is your feature request related to a problem? Please describe.
I'm always frustrated when I click on "Generate test for function" and see "No tests were generated for ...".

Describe the solution you'd like
I would like to see a reason of why there was no tests were generated. For example - "test for ... already exist".

Additional context
It would be perfect to see a location of generated tests if there is already existing one. Or after successful generation.

n1lesh · 2021-07-13T20:47:09Z

@stamblerre I would like to work on this if no one's working.

stamblerre · 2021-07-21T17:05:38Z

Sounds good! /cc @hyangah to confirm that this would be a good thing to work on

n1lesh · 2021-07-21T18:45:25Z

Please correct me if I'm wrong, to generate tests vscode-go calls gotests lib internally as a child process.

However, gotests doesn't return any error if test(s) don't get generated other than "No tests were generated for...". The generateTest method (from gotests) ignores the func if test is already generated for it, and returns no error.

Probably modifying gotests to return an error for existing tests would be a better idea?

stamblerre · 2021-07-23T17:56:08Z

@n1lesh: Thanks for digging into the underlying cause! An alternative option may actually be to move the test generation logic into gopls itself, so that we can adjust the behavior more easily. Would you be willing to work on a gopls change that adds a command to generate tests instead? I know that's a bigger ask than this fix, but I'd be happy to offer pointers if you're interested.

n1lesh · 2021-07-23T18:06:18Z

@stamblerre I can work on the gopls change.

h9jiang · 2024-10-07T20:57:49Z

The Gopls will handle the add unit tests request following steps below:

Gopls respond to code action request and add three new buttons including add unit test for selected functions, functions in this file and functions in this package. The implementation of those three buttons are another LSP rpc call add_unit_test to gopls.

The add unit test for functions button will only show up when the select range overlap with a top level (file level not inline) function/method declaration.

When user click on the button and request for unit test generation, the add_unit_test executeCommand rpc will be invoked and gopls will return WorkspaceEdit as response.
As documented, WorkspaceEdit allows us to return TextDocumentEdit | CreateFile. Gopls will return CreateFile only if the test file does not exist already and will follow the right sequence (create file, then edit the text).

NOTE:

The screenshot is only for demonstration purposes (is my local experiment).
The test file name will follow the convention FILE.go -> FILE_test.go.
The test name will follow the same convention as gotest. See method TestName defined in gotest repo for detail implementation.
For un-exported function, the test function name is Test_function(t *testing.T).
For exported function, the test function name is TestFunction(t *testing.T)
For un-exported struct's method. Test_foo_function(t *testing.T) and Test_foo_Function(t *testing.T)
For exported struct's method. TestFoo_function(t *testing.T) and TestFoo_Function(t *testing.T)
The receiver's type (pointer or struct) does not make any difference to the test function name.

@hyangah @adonovan @findleyr

findleyr · 2024-10-07T21:17:43Z

Cool! A few notes:

Gopls respond to code action request and add three new buttons including add unit test

Let's just say "Add unit test for X". No need to prefix the code action with "Source:", since it is in the "Source Action" section.

When user click on the button and request for unit test generation, the add_unit_test grpc will be invoked and gopls will return WorkspaceEdit as response.

s/grpc/rpc. Also, the command does not return a workspace edit. Rather it calls the workspace/applyEdit server->client rpc.

Test function naming LGTM.

It may be simpler to start with just "Add unit test for function".

adonovan · 2024-10-07T21:18:29Z

Excited to use this! A couple of suggestions:

The wording should be "Add" (since there may already be one), should avoid "unit", and should name the symbol. No "Source:" prefix is needed. So: Add test for function F or Add test for method T.M.
It should only pop up if the symbol is exported. We shouldn't encourage writing tests of internals.

h9jiang · 2024-10-18T15:23:37Z

Add a source code action for single function. if the selected range is enclosed by a function decl or a method decl, we will provide this source code action to Add test for FUNCTION_NAME

Right now, the result of this action is returning a fake error.

findleyr · 2024-10-18T18:22:56Z

Cool! Should probably be "Add a test for Bar", but we can discuss that on the CL. This will be great.

gopherbot · 2024-10-18T18:37:40Z

Change https://go.dev/cl/621056 mentions this issue: gopls/internal/golang: add source code action for add test

gopherbot · 2024-10-18T23:26:23Z

Change https://go.dev/cl/621096 mentions this issue: gopls/internal/golang: create test file if not exist

h9jiang · 2024-10-21T17:47:44Z

Next CL: Following the existing implementation from gotest. When adding for a method, first declare a struct have all the fields as the receiver type.

For embedding type, we will choose the type name as fields name. E.g. Field Node for type ast.Node since struct bar does not provide a name for field which have type ast.Node.

func Test_foo_String(t *testing.T) {
	type fields struct {                  // <----- fields declaration.                
		Node       ast.Node 
		barinside  barinside
		inside     int
		insidecopy int
	}              
        ...
...

type bar struct {
	ast.Node
	barinside
	inside     int
	insidecopy int
}

type foo bar

func (*foo) String() string {return ""}

The end goal of the test looks below. (This is generate by the gotest)

func Test_foo_String(t *testing.T) {
	type fields struct {                          
		Node       ast.Node
		barinside  barinside
		inside     int
		insidecopy int
	}
	tests := []struct {
		name   string
		fields fields
		want   string
	}{
		// TODO: Add test cases.
	}
	for _, tt := range tests {
		t.Run(tt.name, func(t *testing.T) {
			f := &foo{
				Node:       tt.fields.Node,
				barinside:  tt.fields.barinside,
				inside:     tt.fields.inside,
				insidecopy: tt.fields.insidecopy,
			}
			if got := f.String(); got != tt.want {
				t.Errorf("foo.String() = %v, want %v", got, tt.want)
			}
		})
	}
}

gopherbot · 2024-10-21T18:01:37Z

Change https://go.dev/cl/621675 mentions this issue: gopls/internal/golang: add type decl including receiver's type fields

h9jiang · 2024-10-22T19:24:54Z

To update this issue with some decisions:

If the FILE_test.go does not exist, we will generate FILE_test.go with package PACKAGE_test. This follows the best practices where we want to test only the public APIs.
If the FILE_test.go does exist, we will generate test based on the specified package. If the package name is PACKAGE, we will allow generating un-exported functions or methods. If the package name is PACKAGE_test, we will generate only exported functions or methods. If the package name is not PACKAGE or PACKAGE_test, we will not generate any test for any functions/methods.
When adding test for a method, we will try to figure out the constructor instead of composite literals. For T's method, we will try to find any function that match one of the following signature (only allow if the constructor returns at most two values)

func NewT(....) T // returns T
func NewT(....) (T, any) // returns T, any 
func NewT(....) *T // returns *T
func NewT(....) (*T, any) // returns *T, any

If there are multiple return values from the constructor, we can try our best to handle if the second return value is error t, err := NewT(...) otherwise, drop using t, _ := NewT(...)
When adding test for PACKAGE, we will allow constructor newT.
After getting the return value from the functions/methods, we will leave the comparison to the user. The comparison is entirely determined by the context, we can never find an ideal comparison function between reflect.DeepEqual, ==, cmp.Diff. We will leave a comment and ask the user to write the comparison function. (except for the error return type, will explain in the next point)

      // TODO: update the condition below to compare with test.want.
      if true {
        t.Errorf("Foo(%d) = %s, want %s", test.in, got, test.want)
      }

If the function/method returns an error, there are some special handling. The test struct will have a field wantErr bool and correspondingly, a comparison block will be created to make sure the wantErr behavior matches with the returned error.

  got, got1, gotErr := Foo()
  if (err != nil) != test.wantErr {
    t.Errorf("Foo() got err %v, want error: %t", gotErr, test.wantErr)
  }

@hyangah @findleyr @adonovan

gopherbot · 2024-10-25T19:40:43Z

Change https://go.dev/cl/622320 mentions this issue: gopls/internal/golang: create imports map honor x_test.go over x.go

This CL is some glue code which build the connection between the LSP "code action request" with second call which compute the actual DocumentChange. AddTest source code action will create a test file if not already exist and insert a random function at the end of the test file. For testing, an internal boolean option "addTestSourceCodeAction" is created and only effective if set explicitly in marker test. For golang/vscode-go#1594 Change-Id: Ie3d9279ea2858805254181608a0d5103afd3a4c6 Reviewed-on: https://go-review.googlesource.com/c/tools/+/621056 Reviewed-by: Robert Findley <[email protected]> Reviewed-by: Alan Donovan <[email protected]> LUCI-TryBot-Result: Go LUCI <[email protected]>

gopherbot · 2024-11-01T18:25:48Z

Change https://go.dev/cl/623997 mentions this issue: gopls/internal/golang: add special handling for last return error

gopherbot · 2024-11-04T18:59:41Z

Change https://go.dev/cl/621816 mentions this issue: gopls/internal/golang: run testcases as subtests

This CL interpret the user's intention by reading the x_test.go's package name. If the x_test.go exist and is using package name x, we allow adding test for unexported function/method. If the x_test.go does not exist or it exist but using package name x_test, we will only allow adding test for exported function/methods. For golang/vscode-go#1594 Change-Id: I20f6c41dc4c53bb816b40982a0ebbbcb1e3a92e7 Reviewed-on: https://go-review.googlesource.com/c/tools/+/621675 Reviewed-by: Robert Findley <[email protected]> LUCI-TryBot-Result: Go LUCI <[email protected]>

Add test for method is not fully implemented yet. For golang/vscode-go#1594 Change-Id: I4e18183baf96242c209e31e02a1b5cd642844c1d Reviewed-on: https://go-review.googlesource.com/c/tools/+/621057 Reviewed-by: Robert Findley <[email protected]> LUCI-TryBot-Result: Go LUCI <[email protected]>

This commit improves qualifier by consolidating imports from both the main file (x.go) and its corresponding test file (x_test.go). An imports map is used to track all import paths and their local renames. Imports from x_test.go are prioritized over x.go as gopls is generating test in x_test.go. This ensures that the generated qualifier correctly reflects any necessary renames, improving accuracy and consistency. For golang/vscode-go#1594 Change-Id: I457d5f22f7de4fe86006b57487f243494c8e7f6f Reviewed-on: https://go-review.googlesource.com/c/tools/+/622320 LUCI-TryBot-Result: Go LUCI <[email protected]> Reviewed-by: Robert Findley <[email protected]> Auto-Submit: Hongxiang Jiang <[email protected]>

This commit improves the handling of test cases for functions that return an error as their last return value. - A boolean field called "wantErr" is added to the test case struct to indicate whether an error is expected. - The error return value is now handled before comparing other return values, ensuring correct error handling in test cases. For golang/vscode-go#1594 Change-Id: Ib48f6b85b2fdad96cb7fb563b8bdbf17d692f569 Reviewed-on: https://go-review.googlesource.com/c/tools/+/623997 Reviewed-by: Robert Findley <[email protected]> LUCI-TryBot-Result: Go LUCI <[email protected]>

Test case names displayed in subtest names instead of test failure messages. For golang/vscode-go#1594 Change-Id: Iab207cd9af0af99a036dc69a8b2230f12bf876fb Reviewed-on: https://go-review.googlesource.com/c/tools/+/621816 LUCI-TryBot-Result: Go LUCI <[email protected]> Reviewed-by: Robert Findley <[email protected]>

gopherbot · 2024-11-05T19:04:03Z

Change https://go.dev/cl/620696 mentions this issue: gopls/internal/golang: support generating test for method w/o constructor

gopherbot · 2024-11-07T21:36:13Z

Change https://go.dev/cl/620697 mentions this issue: gopls/internal/golang: add missing imports in foo_test.go

Gopls loops over all the function and find the ones qualifies based on its signature. - When a qualifying constructor is found, the test skeleton uses the constructor to initialize the receiver and call the method. - When no constructor is found, gopls generates a test skeleton with a variable declaration using the receiver type (e.g., "var t T") and includes a TODO comment to remind users to implement receiver variable initialization. For golang/vscode-go#1594 Change-Id: I2a703bbd099f03fd1bf85e516f86484805b4a0ae Reviewed-on: https://go-review.googlesource.com/c/tools/+/620696 Auto-Submit: Hongxiang Jiang <[email protected]> LUCI-TryBot-Result: Go LUCI <[email protected]> Reviewed-by: Robert Findley <[email protected]>

gopherbot · 2024-11-11T23:12:05Z

Change https://go.dev/cl/626537 mentions this issue: gopls/internal/golang, go/ssa: remove unnamed input parameter

gopherbot · 2024-11-12T22:07:09Z

Change https://go.dev/cl/627355 mentions this issue: gopls/internal/golang: special handling for input context.Context

- Gopls will honor any renaming of package "testing" if any. - Gopls will collect all the package that have not been imported in foo_test.go and modify the foo_test.go imports. For golang/vscode-go#1594 Change-Id: Id6b87b6417a26f8e925582317e91fb4ebff4a0e7 Reviewed-on: https://go-review.googlesource.com/c/tools/+/620697 Reviewed-by: Robert Findley <[email protected]> LUCI-TryBot-Result: Go LUCI <[email protected]>

- Unnamed/blank parameters are now assigned zero values, as they are not referenced within the function body. - The `zeroString` function has been moved from `go/ssa` to `internal/typesinternal` for better organization. - Honor the input parameter name from the function signature. - Input parameters from both consutrctor and target functions are flattened into the test case struct. Potential field name duplication is handled by introducing prefixes. For golang/vscode-go#1594 Change-Id: I8b56d8f3e0f0432d4f9fe269cc7ba86ea46decfc Reviewed-on: https://go-review.googlesource.com/c/tools/+/626537 Reviewed-by: Robert Findley <[email protected]> Reviewed-by: Alan Donovan <[email protected]> Auto-Submit: Hongxiang Jiang <[email protected]> LUCI-TryBot-Result: Go LUCI <[email protected]>

- Read and determine whether the first input parameter of function or constructor is a context.Context through package name and name comparison. - Call the function or constructor with context.Background() but honor is there is any renaming in foo.go or foo_test.go. - Fix the issue where the constructor param is added to function or method call. For golang/vscode-go#1594 Change-Id: Ic1d145e65bc4b7cb34f637bab8ebdeccd36a33f9 Reviewed-on: https://go-review.googlesource.com/c/tools/+/627355 Reviewed-by: Alan Donovan <[email protected]> Auto-Submit: Hongxiang Jiang <[email protected]> Reviewed-by: Robert Findley <[email protected]> LUCI-TryBot-Result: Go LUCI <[email protected]>

gopherbot · 2024-11-19T23:11:17Z

Change https://go.dev/cl/629756 mentions this issue: gopls/internal/golang: preserve copyright and build constraint

gopherbot · 2024-11-20T20:44:14Z

Change https://go.dev/cl/630119 mentions this issue: gopls/doc/features: add feature documentation for source.addTest

- This commit updates addTest and extractToFile to preserve copyright and build constraint comments when creating new files. - The change introduces utility functions to extract these comments from an ast.File. For golang/vscode-go#1594 Change-Id: I2b2f70d6d4de662c8357bca8558c094496c8b2e9 Reviewed-on: https://go-review.googlesource.com/c/tools/+/629756 Reviewed-by: Alan Donovan <[email protected]> Auto-Submit: Hongxiang Jiang <[email protected]> Reviewed-by: Robert Findley <[email protected]> LUCI-TryBot-Result: Go LUCI <[email protected]>

gopherbot · 2024-11-21T19:05:24Z

Change https://go.dev/cl/629978 mentions this issue: gopls/internal/golang: improve test package name selection for new file

This commit refines the logic for determining the package name of new test files in gopls. Previously, when creating a new test file, gopls blindly choose x_test package. This commit expands the logic to consider the target function's signature. For golang/vscode-go#1594 Change-Id: Ia78003bf007479e48861ce643c8c7c366ff960a3 Reviewed-on: https://go-review.googlesource.com/c/tools/+/629978 Auto-Submit: Hongxiang Jiang <[email protected]> Reviewed-by: Robert Findley <[email protected]> Reviewed-by: Alan Donovan <[email protected]> LUCI-TryBot-Result: Go LUCI <[email protected]>

- Add feature documentation and demo screenshot. - Remove the internal feature gates guarding add test feature. - Replace spaces with tabs in test function body template. - Run format for generated test function body. For golang/vscode-go#1594 Change-Id: Id2106654c24da61ad6aa31723908e4931aa54430 Reviewed-on: https://go-review.googlesource.com/c/tools/+/630119 LUCI-TryBot-Result: Go LUCI <[email protected]> Reviewed-by: Robert Findley <[email protected]> Auto-Submit: Hongxiang Jiang <[email protected]> Reviewed-by: Alan Donovan <[email protected]>

gopherbot · 2024-12-02T17:42:43Z

Change https://go.dev/cl/632857 mentions this issue: gopls/internal/golang: Disable test generation for generic functions

For golang/vscode-go#1594 Change-Id: I8563e4cde1606a23a402bc2154b1af69cb9ff724 Reviewed-on: https://go-review.googlesource.com/c/tools/+/632857 LUCI-TryBot-Result: Go LUCI <[email protected]> Reviewed-by: Robert Findley <[email protected]>

Gopls loops over all the function and find the ones qualifies based on its signature. - When a qualifying constructor is found, the test skeleton uses the constructor to initialize the receiver and call the method. - When no constructor is found, gopls generates a test skeleton with a variable declaration using the receiver type (e.g., "var t T") and includes a TODO comment to remind users to implement receiver variable initialization. For golang/vscode-go#1594 Change-Id: I2a703bbd099f03fd1bf85e516f86484805b4a0ae Reviewed-on: https://go-review.googlesource.com/c/tools/+/620696 Auto-Submit: Hongxiang Jiang <[email protected]> LUCI-TryBot-Result: Go LUCI <[email protected]> Reviewed-by: Robert Findley <[email protected]>

- Gopls will honor any renaming of package "testing" if any. - Gopls will collect all the package that have not been imported in foo_test.go and modify the foo_test.go imports. For golang/vscode-go#1594 Change-Id: Id6b87b6417a26f8e925582317e91fb4ebff4a0e7 Reviewed-on: https://go-review.googlesource.com/c/tools/+/620697 Reviewed-by: Robert Findley <[email protected]> LUCI-TryBot-Result: Go LUCI <[email protected]>

- Unnamed/blank parameters are now assigned zero values, as they are not referenced within the function body. - The `zeroString` function has been moved from `go/ssa` to `internal/typesinternal` for better organization. - Honor the input parameter name from the function signature. - Input parameters from both consutrctor and target functions are flattened into the test case struct. Potential field name duplication is handled by introducing prefixes. For golang/vscode-go#1594 Change-Id: I8b56d8f3e0f0432d4f9fe269cc7ba86ea46decfc Reviewed-on: https://go-review.googlesource.com/c/tools/+/626537 Reviewed-by: Robert Findley <[email protected]> Reviewed-by: Alan Donovan <[email protected]> Auto-Submit: Hongxiang Jiang <[email protected]> LUCI-TryBot-Result: Go LUCI <[email protected]>

- Read and determine whether the first input parameter of function or constructor is a context.Context through package name and name comparison. - Call the function or constructor with context.Background() but honor is there is any renaming in foo.go or foo_test.go. - Fix the issue where the constructor param is added to function or method call. For golang/vscode-go#1594 Change-Id: Ic1d145e65bc4b7cb34f637bab8ebdeccd36a33f9 Reviewed-on: https://go-review.googlesource.com/c/tools/+/627355 Reviewed-by: Alan Donovan <[email protected]> Auto-Submit: Hongxiang Jiang <[email protected]> Reviewed-by: Robert Findley <[email protected]> LUCI-TryBot-Result: Go LUCI <[email protected]>

gopherbot · 2024-12-04T19:46:23Z

Change https://go.dev/cl/633703 mentions this issue: gopls/internal/golang: Disable test generation for generic functions

gopherbot added this to the Untriaged milestone Jun 28, 2021

stamblerre added the FeatureRequest label Jun 28, 2021

stamblerre modified the milestones: Untriaged, Backlog Jun 28, 2021

stamblerre changed the title ~~Add info to "No tests generated for" message~~ Add feedback if test generation fails Jun 28, 2021

stamblerre added the HelpWanted Issues that are not prioritized by the maintainers. Help is requested from community contributors. label Jun 28, 2021

hyangah changed the title ~~Add feedback if test generation fails~~ tools: migrate test generation features to gopls Jul 28, 2021

hyangah mentioned this issue Jul 28, 2021

tools: reduce the number of 3rd party tools the extension requires #1652

Open

9 tasks

n1lesh mentioned this issue Jul 30, 2021

x/tools/gopls: migrate "test generation" features to gopls golang/go#47479

Closed

suzmue mentioned this issue Dec 3, 2021

Generate unit test for functions with variable semantics #1927

Closed

suzmue self-assigned this Jan 11, 2022

hyangah removed the HelpWanted Issues that are not prioritized by the maintainers. Help is requested from community contributors. label Feb 8, 2022

suzmue removed their assignment Feb 8, 2022

hyangah mentioned this issue Jun 20, 2024

"Go: Generate Unit Tests For Function" does not handle generics #3425

Closed

hyangah assigned h9jiang Oct 2, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

tools: migrate test generation features to `gopls` #1594

tools: migrate test generation features to `gopls` #1594

emil14 commented Jun 28, 2021

n1lesh commented Jul 13, 2021

stamblerre commented Jul 21, 2021

n1lesh commented Jul 21, 2021 •

edited

Loading

stamblerre commented Jul 23, 2021

n1lesh commented Jul 23, 2021 •

edited

Loading

h9jiang commented Oct 7, 2024 •

edited by hyangah

Loading

findleyr commented Oct 7, 2024 •

edited

Loading

adonovan commented Oct 7, 2024

h9jiang commented Oct 18, 2024

findleyr commented Oct 18, 2024

gopherbot commented Oct 18, 2024

gopherbot commented Oct 18, 2024

h9jiang commented Oct 21, 2024 •

edited

Loading

gopherbot commented Oct 21, 2024

h9jiang commented Oct 22, 2024 •

edited

Loading

gopherbot commented Oct 25, 2024

gopherbot commented Nov 1, 2024

gopherbot commented Nov 4, 2024

gopherbot commented Nov 5, 2024

gopherbot commented Nov 7, 2024

gopherbot commented Nov 11, 2024

gopherbot commented Nov 12, 2024

gopherbot commented Nov 19, 2024

gopherbot commented Nov 20, 2024

gopherbot commented Nov 21, 2024

gopherbot commented Dec 2, 2024

gopherbot commented Dec 4, 2024

tools: migrate test generation features to gopls #1594

tools: migrate test generation features to gopls #1594

Comments

emil14 commented Jun 28, 2021

n1lesh commented Jul 13, 2021

stamblerre commented Jul 21, 2021

n1lesh commented Jul 21, 2021 • edited Loading

stamblerre commented Jul 23, 2021

n1lesh commented Jul 23, 2021 • edited Loading

h9jiang commented Oct 7, 2024 • edited by hyangah Loading

findleyr commented Oct 7, 2024 • edited Loading

adonovan commented Oct 7, 2024

h9jiang commented Oct 18, 2024

findleyr commented Oct 18, 2024

gopherbot commented Oct 18, 2024

gopherbot commented Oct 18, 2024

h9jiang commented Oct 21, 2024 • edited Loading

gopherbot commented Oct 21, 2024

h9jiang commented Oct 22, 2024 • edited Loading

gopherbot commented Oct 25, 2024

gopherbot commented Nov 1, 2024

gopherbot commented Nov 4, 2024

gopherbot commented Nov 5, 2024

gopherbot commented Nov 7, 2024

gopherbot commented Nov 11, 2024

gopherbot commented Nov 12, 2024

gopherbot commented Nov 19, 2024

gopherbot commented Nov 20, 2024

gopherbot commented Nov 21, 2024

gopherbot commented Dec 2, 2024

gopherbot commented Dec 4, 2024

tools: migrate test generation features to `gopls` #1594

tools: migrate test generation features to `gopls` #1594

n1lesh commented Jul 21, 2021 •

edited

Loading

n1lesh commented Jul 23, 2021 •

edited

Loading

h9jiang commented Oct 7, 2024 •

edited by hyangah

Loading

findleyr commented Oct 7, 2024 •

edited

Loading

h9jiang commented Oct 21, 2024 •

edited

Loading

h9jiang commented Oct 22, 2024 •

edited

Loading