Agent is prompted to verify remote server authenticity when cloning submodule from unkown host

[regner]

Buildkite Agent Version:

$ buildkite-agent -v
buildkite-agent version 3.0-beta.28, build 1618

When cloning a Git repo from BitBucket that has submodules in GitHub the agent is prompted to add the remote host to the known hosts:

Cloning into 'redacted'...
The authenticity of host 'github.com (192.30.255.112)' can't be established.
RSA key fingerprint is 16:27:ac:a5:76:28:2d:36:63:1b:56:4d:eb:df:a6:48.
Are you sure you want to continue connecting (yes/no)?

Most of our repos are in BitBucket and cloning those works fine. It appears the agent preemptively adds the remote host to the known hosts (https://github.com/buildkite/agent/blob/master/bootstrap/bootstrap.go#L972) and things are grand. However it doesn't add submodule hosts to the known hosts. I assume if on that agent machine we had a pipeline that used GitHub for the repo, and not the submodule, that would work and then this pipeline would pass.

[regner]

For now I have just manually added the GitHub host to the agents known hosts.

[lox]

Implemented! Will get this into a beta soon.

[regner]

Thanks for the quick fix. :)

[jam13]

I seem to be hitting this problem on latest buildkite agent (3.4.0).

I've got a Bitbucket repo that has a single submodule from GitHub. When it tries to update the submodules it prompts to accept the GitHub keys and so prevents the build from continuing.

It doesn't fail on every build because an agent instance might have added the GitHub keys to wellknown through another pipeline job, but on newly deployed instances (AWS stack) it fails every time.

Looking at the code, the problem seems to be that it uses the following command to get the submodule URLs which it then uses to update the known_hosts:

git submodule foreach --recursive git ls-remote --get-url

As this is run before the submodules are initialised, the foreach always returns empty and so no URLs are returned.