Don't use storage account key for object storage if SP or MI is provided #111

yvespp · 2021-11-18T12:37:03Z

The plugin no longer uses the storage access key to access the blobs by default. The key is only used if it's configured via AZURE_STORAGE_ACCOUNT_ACCESS_KEY. Therefore "Allow storage account key access" can be disabled on the storage account if using MI or SP.

Changes:

Switched object_store to the newer azblobclient
When the storage account access key is not provided delegation SAS instead of service SAS is used.
- delegation SAS isn't implemented in the azblob client yet so I implemented a simple version in delegationsas.go. This code can be delete once it's implemented in azblob
- resourceGroupConfigKey and subscriptionIDConfigKey are no longer needed for the object store
Added an integration test for the object_storage
- Only runs when the tag integration is specified e.g. go test ./... -tags=integration -v
- The storage account for the test has to be created separatly.
Updated README
Updated deps
- go 1.17
- go dependencies
- busybox image

sseago

Just a question around the golang version for now -- I'll look at this in more detail a bit later.

sseago · 2021-11-18T14:34:52Z

.github/workflows/push.yml

@@ -13,10 +13,10 @@ jobs:
    runs-on: ubuntu-latest
    steps:

-    - name: Set up Go 1.14
+    - name: Set up Go 1.17


velero still uses go 1.16 on the main branch. We should probably do the same here unless there is a specific need for 1.17 for this plugin.

sseago · 2021-11-18T14:35:12Z

Dockerfile

@@ -12,12 +12,12 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.

-FROM golang:1.13-buster AS build
+FROM golang:1.17-buster AS build


As above, velero still uses go 1.16 on the main branch. We should probably do the same here unless there is a specific need for 1.17 for this plugin.

yvespp · 2021-11-19T06:55:31Z

I downgraded it to 1.16 and also added a go setup step to pr.yaml, it seems it was using the preinstalled version before.

yvespp · 2021-11-26T12:37:13Z

Hi!

Has someone time to review this?
BTW: is there a way to run integration tests against Azure in this repo?

Thanks,
Yves

yvespp · 2022-02-16T12:19:06Z

Rebased to master.
Any chance this can be merged?

nxf5025 · 2022-02-17T22:03:04Z

This is exactly what we need as well and makes a lot more sense than dealing with access keys when they aren't necessary.

codecov-commenter · 2022-06-29T11:31:26Z

Codecov Report

Merging #111 (e7efb9e) into main (73d1530) will increase coverage by 1.29%.
The diff coverage is 7.00%.

@@            Coverage Diff             @@
##             main     #111      +/-   ##
==========================================
+ Coverage   12.77%   14.07%   +1.29%     
==========================================
  Files           4        4              
  Lines         626      668      +42     
==========================================
+ Hits           80       94      +14     
- Misses        542      571      +29     
+ Partials        4        3       -1

Impacted Files	Coverage Δ
velero-plugin-for-microsoft-azure/object_store.go	`2.38% <0.54%> (-0.27%)`	⬇️
...o-plugin-for-microsoft-azure/volume_snapshotter.go	`23.68% <4.44%> (+0.28%)`	⬆️
velero-plugin-for-microsoft-azure/common.go	`24.59% <55.55%> (+24.59%)`	⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

vzabawski · 2022-07-14T11:39:56Z

Does the PR also addresses this issue? vmware-tanzu/velero#4930

yvespp · 2022-07-14T14:30:39Z

Does the PR also addresses this issue? vmware-tanzu/velero#4930

Yes, ListKeys will be removed with this PR.

vzabawski · 2022-07-15T07:14:47Z

Awesome! Thank you for your contribution.

chrisob · 2022-07-28T12:05:04Z

Reliance on Azure storage account shared keys is preventing my org from using Velero, would be great to see this merged ❤️ @ywk253100 @eleanor-millman

anshulahuja98 · 2022-09-08T05:14:02Z

HI @yvespp
I was trying to test out this PR
1 basic flow I was not able to validate - I created a BSL with incorrect params(SA didn't exist)
But the BSL is showing as available. Possibly some underlying layers are not returning errors?
The logs also did not help much debugging

When I created a backup, the following error came in the Backup Status. The BSL was still showing as available

│ completionTimestamp: "2022-09-08T05:12:03Z" ││ expiration: "2022-10-08T05:12:03Z" ││ failureReason: | ││ error checking if backup already exists in object storage: rpc error: code = Unknown desc = ===== RESPONSE ERROR (ErrorCode=AuthorizationPermissionMismatch) ===== ││ Description=, Details: (none) ││ formatVersion: 1.1.0 ││ phase: Failed ││ startTimestamp: "2022-09-08T05:12:03Z" ││ version: 1

velero-plugin-for-microsoft-azure/object_store.go

go.mod

ywk253100 · 2022-09-09T09:32:16Z

@yvespp Sorry for the late review.

I have added some comments but haven't reviewed the delegationsas.go yet and I will continue to add comments.

mimmi-gjems · 2022-09-22T09:07:22Z

Hi! Would be really great if this could be merged sometime soon :) @sseago @ywk253100 @dsu-igeek

vzabawski · 2022-09-22T11:28:49Z

I second that. This PR will allow me to start using Velero.

yvespp · 2022-09-26T07:01:33Z

HI @yvespp I was trying to test out this PR 1 basic flow I was not able to validate - I created a BSL with incorrect params(SA didn't exist) But the BSL is showing as available. Possibly some underlying layers are not returning errors? The logs also did not help much debugging

When I created a backup, the following error came in the Backup Status. The BSL was still showing as available

│ completionTimestamp: "2022-09-08T05:12:03Z" ││ expiration: "2022-10-08T05:12:03Z" ││ failureReason: | ││ error checking if backup already exists in object storage: rpc error: code = Unknown desc = ===== RESPONSE ERROR (ErrorCode=AuthorizationPermissionMismatch) ===== ││ Description=, Details: (none) ││ formatVersion: 1.1.0 ││ phase: Failed ││ startTimestamp: "2022-09-08T05:12:03Z" ││ version: 1

Hi @anshulahuja98, do you know how the backup location is verified? I was looking through the code but couldn't find it.

README.md

changelogs/unreleased/000-yvespp

ywk253100 · 2023-01-09T01:36:35Z

@yvespp I added a few comments about the document, please take a look.
And please also resolve the conflicts, rebase and squash the commits

Switches the to newer blob client Add integration test Signed-off-by: Yves Peter <[email protected]>

Signed-off-by: Yves Peter <[email protected]>

yvespp · 2023-01-09T12:56:10Z

@yvespp I added a few comments about the document, please take a look. And please also resolve the conflicts, rebase and squash the commits

@ywk253100 I addressed your comments, rebased and squashed.

ywk253100 · 2023-01-10T02:33:33Z

The changes have passed the end-to-end testing running locally. Thanks @yvespp

@sseago Could you take a look at this PR when you are available?

jkroepke · 2023-02-08T09:52:51Z

Hi all, could the PR released? It looks like this PR is not port of the 1.6.1 version.

Current workaround: Add Reader and Data Access Role in addition to Storage Blob Data Contributor on the storage account to avoid the error ... does not have authorization to perform action 'Microsoft.Storage/storageAccounts/listKeys/action' over scope ...

yvespp · 2023-02-15T08:07:23Z

Hi all, could the PR released? It looks like this PR is not port of the 1.6.1 version.

Current workaround: Add Reader and Data Access Role in addition to Storage Blob Data Contributor on the storage account to avoid the error ... does not have authorization to perform action 'Microsoft.Storage/storageAccounts/listKeys/action' over scope ...

I think the plan is to release this with the next minor version 1.7.0 and Velero 1.11: #111 (comment)

If you want to try it out you can use my image which includes this pull request: https://hub.docker.com/r/yvespp/velero-plugin-for-microsoft-azure
It's still on 1.6.0 though.

ywk253100 · 2023-02-15T09:13:00Z

@yvespp Thanks
@jkroepke The fix will be available in 1.7. And as the changes introduced by this PR are quite fundamental, we'll not back-port the changes to 1.6

We introduced changes in vmware-tanzu#111 to remove the logic of listing storage account access key, the Velero Azure plugin supports auth via Azure AD directly after the changes, but that isn't enough as Restic/Kopia still doesn't support auth via Azure AD at this moment, this will cause filesystem backup failure on Azure. So we revert the doc change in this commit and Velero still needs the permission of listing storage access key to work as expected. But as we keep the code changes, users can workaround the permission issue by refer to vmware-tanzu/velero#5984 Signed-off-by: Wenkai Yin(尹文开) <[email protected]>

Revert the changes made to doc in #111

vikrantoct7 · 2023-03-21T18:08:07Z

If this PR is already merged, may i know the plugin version in which this PR is available?

mkemmerz · 2023-04-04T12:39:52Z

Hello everyone - is there any release date planned for this feature yet? This feature is essential for Azure and I would like to use it :)

ywk253100 · 2023-04-06T09:53:19Z

@vikrantoct7 @mkemmerz This change isn't available in Velero 1.11, please see vmware-tanzu/velero#5984 for details.

But if you guys take backup without Restic/Kopia, the Velero Azure plugin can be configured as work without listing storage account access keys.

github-actions bot assigned yvespp Nov 18, 2021

github-actions bot requested review from dsu-igeek and sseago November 18, 2021 12:37

sseago reviewed Nov 18, 2021

View reviewed changes

ywk253100 self-requested a review November 19, 2021 03:11

yvespp force-pushed the delegation_sas branch from 0838c8a to b0c47ff Compare November 19, 2021 06:52

yvespp force-pushed the delegation_sas branch from c8a7d4f to eda23c5 Compare December 3, 2021 10:11

yvespp force-pushed the delegation_sas branch 2 times, most recently from 34066b0 to b43a9b8 Compare February 16, 2022 10:13

yvespp force-pushed the delegation_sas branch from b43a9b8 to 503f3f2 Compare April 22, 2022 08:40

yvespp mentioned this pull request Apr 22, 2022

Azure: don't use storage account key for object storage if SP or MI is provided vmware-tanzu/velero#4267

Closed

yvespp mentioned this pull request May 4, 2022

azblob - implement UserDelegationCredential Azure/azure-sdk-for-go#16916

Closed

yvespp force-pushed the delegation_sas branch from 503f3f2 to 48f795e Compare June 29, 2022 11:27

ywk253100 reviewed Sep 9, 2022

View reviewed changes

yvespp force-pushed the delegation_sas branch 2 times, most recently from 5a45768 to 96b6d31 Compare September 22, 2022 18:09

ywk253100 reviewed Jan 9, 2023

View reviewed changes

README.md Outdated Show resolved Hide resolved

ywk253100 reviewed Jan 9, 2023

View reviewed changes

README.md Outdated Show resolved Hide resolved

ywk253100 reviewed Jan 9, 2023

View reviewed changes

changelogs/unreleased/000-yvespp Outdated Show resolved Hide resolved

ywk253100 mentioned this pull request Jan 9, 2023

[Azure] Add AAD Workload Identity support vmware-tanzu/velero#5116

Closed

yvespp added 2 commits January 9, 2023 12:58

Use delegation sas by default

a8ab720

Switches the to newer blob client Add integration test Signed-off-by: Yves Peter <[email protected]>

update busybox, .dockerignore the Dockerfile

e7efb9e

Signed-off-by: Yves Peter <[email protected]>

yvespp force-pushed the delegation_sas branch from cac547b to e7efb9e Compare January 9, 2023 11:59

ywk253100 approved these changes Jan 10, 2023

View reviewed changes

blackpiglet requested review from blackpiglet and removed request for dsu-igeek January 12, 2023 09:05

reasonerjt approved these changes Jan 16, 2023

View reviewed changes

reasonerjt merged commit dc34ed3 into vmware-tanzu:main Jan 16, 2023

ywk253100 mentioned this pull request Feb 17, 2023

Remove the "subscriptionId" and "resourceGroup" items from the BSL config when installing Velero in E2E test vmware-tanzu/velero#5870

Closed

ywk253100 mentioned this pull request Mar 15, 2023

Listing storage account access key is still needed for Restic/Kopia to work as expected on Azure vmware-tanzu/velero#5984

Closed

ywk253100 mentioned this pull request Mar 15, 2023

Revert the changes made to doc in #111 #173

Merged

reasonerjt added a commit that referenced this pull request Mar 19, 2023

Merge pull request #173 from ywk253100/230315_doc

de5a27d

Revert the changes made to doc in #111

vikrantoct7 mentioned this pull request Mar 22, 2023

Install and configure Azure AD Workload Identity adamrushuk/devops-lab#219

Open

9 tasks

ywk253100 mentioned this pull request Mar 27, 2023

docs: fix custom role definition & SP creation #172

Closed

yvespp deleted the delegation_sas branch April 4, 2023 15:13

yvespp restored the delegation_sas branch April 4, 2023 15:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Don't use storage account key for object storage if SP or MI is provided #111

Don't use storage account key for object storage if SP or MI is provided #111

yvespp commented Nov 18, 2021

sseago left a comment

sseago Nov 18, 2021

sseago Nov 18, 2021

yvespp commented Nov 19, 2021

yvespp commented Nov 26, 2021

yvespp commented Feb 16, 2022

nxf5025 commented Feb 17, 2022

codecov-commenter commented Jun 29, 2022 •

edited

Loading

vzabawski commented Jul 14, 2022

yvespp commented Jul 14, 2022

vzabawski commented Jul 15, 2022

chrisob commented Jul 28, 2022

anshulahuja98 commented Sep 8, 2022

ywk253100 commented Sep 9, 2022

mimmi-gjems commented Sep 22, 2022

vzabawski commented Sep 22, 2022

yvespp commented Sep 26, 2022

ywk253100 commented Jan 9, 2023 •

edited

Loading

yvespp commented Jan 9, 2023 •

edited

Loading

ywk253100 commented Jan 10, 2023

jkroepke commented Feb 8, 2023 •

edited

Loading

yvespp commented Feb 15, 2023 •

edited

Loading

ywk253100 commented Feb 15, 2023 •

edited

Loading

vikrantoct7 commented Mar 21, 2023

mkemmerz commented Apr 4, 2023 •

edited

Loading

ywk253100 commented Apr 6, 2023

Don't use storage account key for object storage if SP or MI is provided #111

Don't use storage account key for object storage if SP or MI is provided #111

Conversation

yvespp commented Nov 18, 2021

sseago left a comment

Choose a reason for hiding this comment

sseago Nov 18, 2021

Choose a reason for hiding this comment

sseago Nov 18, 2021

Choose a reason for hiding this comment

yvespp commented Nov 19, 2021

yvespp commented Nov 26, 2021

yvespp commented Feb 16, 2022

nxf5025 commented Feb 17, 2022

codecov-commenter commented Jun 29, 2022 • edited Loading

Codecov Report

vzabawski commented Jul 14, 2022

yvespp commented Jul 14, 2022

vzabawski commented Jul 15, 2022

chrisob commented Jul 28, 2022

anshulahuja98 commented Sep 8, 2022

ywk253100 commented Sep 9, 2022

mimmi-gjems commented Sep 22, 2022

vzabawski commented Sep 22, 2022

yvespp commented Sep 26, 2022

ywk253100 commented Jan 9, 2023 • edited Loading

yvespp commented Jan 9, 2023 • edited Loading

ywk253100 commented Jan 10, 2023

jkroepke commented Feb 8, 2023 • edited Loading

yvespp commented Feb 15, 2023 • edited Loading

ywk253100 commented Feb 15, 2023 • edited Loading

vikrantoct7 commented Mar 21, 2023

mkemmerz commented Apr 4, 2023 • edited Loading

ywk253100 commented Apr 6, 2023

codecov-commenter commented Jun 29, 2022 •

edited

Loading

ywk253100 commented Jan 9, 2023 •

edited

Loading

yvespp commented Jan 9, 2023 •

edited

Loading

jkroepke commented Feb 8, 2023 •

edited

Loading

yvespp commented Feb 15, 2023 •

edited

Loading

ywk253100 commented Feb 15, 2023 •

edited

Loading

mkemmerz commented Apr 4, 2023 •

edited

Loading