io.copy() got rbd: ret=22, Invalid argument

[mysugar]

	image,err := rbd.OpenImage(suite.storage.ioctx, "d1","")
	require.NoError(suite.T(), err)
	defer image.Close()
	lfile,err := os.Create("/tmp/d1.rbd")
	require.NoError(suite.T(), err)
	defer lfile.Close()
	wrriten,err := io.Copy(lfile,image)
	fmt.Println("Get File wrriten ",wrriten ," data to /tmp/d1.rbd")
	require.NoError(suite.T(), err)

Got

Get File wrriten  104857600  data to /tmp/d1.rbd
    TestCephRBDStorageSuite/TestGetFile: ceph_rbd_test.go:56: 
        	Error Trace:	ceph_rbd_test.go:56
        	Error:      	Received unexpected error:
        	            	rbd: ret=22, Invalid argument
        	Test:       	TestCephRBDStorageSuite/TestGetFile
    --- FAIL: TestCephRBDStorageSuite/TestGetFile (0.63s)

my rbd image size is 100M ,and afert this func /tmp/d1.rbd size is also 100M.
any good idea?

[mysugar]

what about change

go-ceph/rbd/rbd.go

Lines 710 to 712 in d4440eb

	if ret < 0 {
	return 0, RBDError(ret)
	}

to

	if ret < 0 {
                size,_ := image.GetSize()
                if uint64(image.offset) == size {
			return 0, io.EOF
		}
		return 0, RBDError(ret)
	}

[nixpanic]

Reading from an RBD image that hits EOF, should return a short read, or 0 in case nothing is read. I do not understand why you would get errno=22: Invalid argument. Unless io.Copy() causes the offset for reading to pass EOF somehow. This needs a little more debugging to understand what is going on, and wht rbd_read() returns an error that we do not expect.

Please provide the following details:

• Ceph version of the client (local librdb.so)
• Ceph version of the cluster (remote OSDs)
• confirmation that your suggestion of the change prevents the problem
• PR with a new test case using io.Copy() (expected to fail CI)

Could you try the following? This could be a workaround:

image, err := rbd.OpenImage(suite.storage.ioctx, "d1","")
lr := io.LimitReader(image, image.GetSize())
writen, err := io.Copy(lfile, lr)

That way, there is an additional LimitReader that prevents going passed image.GetSize().

[mysugar]

Thanks for your workaround! That works!

1.Client version 12.2.13-0ubuntu0.18.04.2
2.Cluster version ceph version 12.2.13 (584a20eb0237c657dc0567da126be145106aa47e) luminous (stable)
3.My suggestion works (of course, i think your solution is better)
4.Test case need further recurrence.

This problem seems related to the image. io.copy() other images with no problem.

[phlogistonjohn]

I'm following up on a few issues. From my reading of this issue it sounds like the problem lies in the ceph libs rather than go-ceph. I also see that you are/were using ceph luminous client libs. If you could I'd like to suggest that you retest with newer ceph versions and if the issue does not reproduce we'll close this out.

I'll also point out that as of our v0.5.0 release, made earlier this week, we're officially deprecating luminous as a build target. In v0.6.0, due in October, building against luminous will be dropped.

[mysugar]

Thank you！The issue did not reproduce.