Minikube VM cannot access anymore to host loopback interface through NAT with VirtualBox 7.x

[cvila84]

What Happened?

By default, VirtualBox 7.x will now prevent newly created VM to access host loopback interface through NAT (with default IP 10.0.2.2).

To overcome this, we need to pass a new parameter --nat-localhostreachable1 at VM creation time (with modifyvm command). Careful as passing this parameter with previous VB versions will fail :-(

VBoxManage modifyvm <uuid | vmname> [--nat-netN= network | default ] [--nat-pfN= [rule-name],tcp | udp,[host-IP],hostport,[guest-IP],guestport ] [--nat-pfN=delete=rule-name] [--nat-tftp-prefixN=prefix] [--nat-tftp-fileN=filename]
      [--nat-tftp-serverN=IP-address] [--nat-bind-ipN=IP-address] [--nat-dns-pass-domainN= on | off ] [--nat-dns-proxyN= on | off ] [--nat-dns-host-resolverN= on | off ] [--nat-localhostreachableN= on | off ]
      [--nat-settingsN=[mtu],[socksnd],[sockrcv],[tcpsnd],[tcprcv]] [--nat-alias-modeN= default | [log],[proxyonly],[sameports] ]

Attach the log file

--- No log file as minikube commands are working, only VM configuration needs to be changed ---

Operating System

Windows

Driver

VirtualBox

[afbjorklund]

Thanks for the information!

Since upstream libmachine is no longer being developed, we will need to patch it (virtualbox) here:

https://github.com/machine-drivers/machine/issues

It should be possible to check VBoxManage --version from the code, before trying to attempt it.

https://github.com/machine-drivers/machine/blob/master/drivers/virtualbox/virtualbox.go#L380

[cvila84]

I tried this but I don't have any environment to build minikube : machine-drivers/machine#39

[afbjorklund]

Fixing it for docker-machine would be perfectly acceptable, but I think I have VirtualBox 6.1

docker-machine create default --driver virtualbox

--virtualbox-boot2docker-url https://github.com/afbjorklund/boot2docker/releases/download/be5929d/boot2docker.iso

The "virtualbox" driver is compiled-in (not external), so the place to change would be go.mod

github.com/docker/machine v0.16.2

github.com/docker/machine => github.com/machine-drivers/machine v0.7.1-0.20211105063445-78a84df85426

[cvila84]

Hello, I was able to build the docker-machine with the PR and tested it successfully with VB7

.\docker-machine-Windows-x86_64.exe --debug create default --driver virtualbox --virtualbox-host-loopback-reachable --virtualbox-boot2docker-url https://github.com/afbjorklund/boot2docker/releases/download/be5929d/boot2docker.iso
Docker Machine Version:  dev, build 619b0218
Found binary path at C:\Users\cvila\dev\Work\workspace\go\src\github.com\machine-drivers\machine\bin\docker-machine-Windows-x86_64.exe
Launching plugin server for driver virtualbox
Plugin server listening at address 127.0.0.1:62324

[...]

(default) Calling .PreCreateCheck
(default) DBG | COMMAND: C:\Program Files\Oracle\VirtualBox\VBoxManage.exe --version
(default) DBG | STDOUT:
(default) DBG | {
(default) DBG | 7.0.4r154605
(default) DBG | }
(default) DBG | STDERR:
(default) DBG | {
(default) DBG | }

[...]

(default) DBG | COMMAND: C:\Program Files\Oracle\VirtualBox\VBoxManage.exe modifyvm default --firmware bios --bioslogofadein off --bioslogofadeout off --bioslogodisplaytime 0 --biosbootmenu disabled --ostype Linux26_64 --cpus 1 --memory 1024 --acpi on --ioapic on --rtcuseutc on --natdnshostresolver1 off --natdnsproxy1 on --cpuhotplug off --pae on --hpet on --hwvirtex on --nestedpaging on --largepages on --vtxvpid on --accelerate3d off --boot1 dvd --natlocalhostreachable1 on
(default) DBG | STDOUT:
(default) DBG | {
(default) DBG | }
(default) DBG | STDERR:
(default) DBG | {
(default) DBG | }

[cvila84]

Hello, I was able to finally build minikube (in minikube) with the PR and tested it successfully with VB7

I used the following go.mod to build minikube

	github.com/docker/machine => github.com/cvila84/machine v0.0.0-20230102153142-619b02185a79

In minikube logs (lastStart.txt), I can see the same I saw with docker-machine

I0104 11:03:36.810829   27572 out.go:177] 😄  minikube v1.28.1 sur Microsoft Windows 10 Enterprise 10.0.19045 Build 19045
I0104 11:03:36.846012   27572 virtualbox.go:136] virtual box version: 7.0.4r154605
I0104 11:03:36.847346   27572 out.go:177] ✨  Utilisation du pilote virtualbox basé sur la configuration de l'utilisateur
I0104 11:03:36.848933   27572 start.go:282] selected driver: virtualbox
I0104 11:03:36.848933   27572 start.go:808] validating driver "virtualbox" against <nil>
I0104 11:03:36.848933   27572 start.go:819] status for virtualbox: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:7.0.4r154605
}
I0104 11:03:49.203845   27572 main.go:134] libmachine: COMMAND: C:\Program Files\Oracle\VirtualBox\VBoxManage.exe createvm --basefolder C:\Users\cvila\.minikube\machines\minikube --name minikube --register
I0104 11:03:49.724685   27572 main.go:134] libmachine: STDOUT:
{
Virtual machine 'minikube' is created and registered.
UUID: dea29da8-1f85-4bbb-9c04-ddff881e616f
Settings file: 'C:\Users\cvila\.minikube\machines\minikube\minikube\minikube.vbox'
}
I0104 11:03:49.724685   27572 main.go:134] libmachine: STDERR:
{
}
I0104 11:03:49.724685   27572 main.go:134] libmachine: VM CPUS: 6
I0104 11:03:49.724685   27572 main.go:134] libmachine: VM Memory: 12288
I0104 11:03:49.724685   27572 main.go:134] libmachine: COMMAND: C:\Program Files\Oracle\VirtualBox\VBoxManage.exe modifyvm minikube --firmware bios --bioslogofadein off --bioslogofadeout off --bioslogodisplaytime 0 --biosbootmenu disabled --ostype Linux26_64 --cpus 6 --memory 12288 --acpi on --ioapic on --rtcuseutc on --natdnshostresolver1 off --natdnsproxy1 off --cpuhotplug off --pae on --hpet on --hwvirtex on --nestedpaging on --largepages on --vtxvpid on --accelerate3d off --boot1 dvd --natlocalhostreachable1 on
I0104 11:03:49.794649   27572 main.go:134] libmachine: STDOUT:
{
}
I0104 11:03:49.794649   27572 main.go:134] libmachine: STDERR:
{
}

Do you need something else from me ?

[afbjorklund]

Sounds great, thanks for the contribution and sorry for the long wait.

Merged now. 48d7306a06225f35b026669fc0c1f7fd0af18ebe

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[cvila84]

/remove-lifecycle stale

as the PR on machine is already merged, and remaining activity (if not already done) is to refer to this latest version in minikube

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

[k8s-ci-robot]

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.