SurrealDB's CBOR usage drives me nuts; help me fix this code?

[senpro-ingwersenk]

Hello!

Thanks to SurrealDB, I got very close up and intimate with CBOR in the form of a million issues in terms of building my structs and debugging stuff - because they use a lot of custom tags. Not too surprising since the spec leaves many unreserved slots - but this makes actually "making" things incredibly difficult...

Basically, when querying a row that references other tables, they will use models.RecordID as the return type. I tried to overwrite this with... well, this abomination:

func (p *Process) UnmarshalCBOR(data []byte) error {
	txtCbr, err := cbor.Diagnose(data)
	fmt.Println("--- Process Unmarshall ---")
	if err != nil {
		fmt.Println("Error")
		fmt.Println(err.Error())
	} else {
		fmt.Println(txtCbr)
	}
	fmt.Println("--- /Process Unmarshall ---")

	// Use a generic map to decode the raw CBOR data
	temp := make(map[string]interface{})

	// Unmarshal the data into the map
	c := models.CborUnmarshaler{}
	if err := c.Unmarshal(data, &temp); err != nil {
		return fmt.Errorf("failed to unmarshal CBOR: %w", err)
	}

	// Manually assign fields from the map to the struct
	for key, value := range temp {
		var ok bool = true
		fmt.Println("... " + key)
		switch key {
		case "title":
			p.Title, ok = value.(string)
		case "description":
			p.Description, ok = value.(string)
		case "created_at":
			customTime, ok := value.(models.CustomDateTime)
			if ok {
				p.CreatedAt = customTime.Time
			}
		case "updated_at":
			customTime, ok := value.(models.CustomDateTime)
			if ok {
				p.UpdatedAt = customTime.Time
			}
		case "storage":
			recordIds, ok := internal.ExtractRecordIDs(value) //.([]interface{})
			spew.Println(value, ok)
			if ok {
				for i := range recordIds {
					spew.Println(recordIds[i])
					record, err := internal.FindModel[storage.Storage](recordIds[i])
					spew.Println(record, err)
					if err == nil {
						p.Storage = append(p.Storage, record)
					} else {
						ok = false
						break
					}
				}
				//p.Storage = internal.RecordIDtoStruct[*storage.Storage](recordIds)
			} else {
				fmt.Println("storage was not ok")
				fmt.Println((err.Error()))
			}
		case "legal_basis":
			recordIds, ok := internal.ExtractRecordIDs(value) //.([]interface{})
			if ok {
				p.LegalBasis = internal.RecordIDtoStruct[*legal_basis.LegalBasis](recordIds)
			}
		case "responsible":
			recordIds, ok := internal.ExtractRecordIDs(value) //.([]interface{})
			if ok {
				p.Responsible = internal.RecordIDtoStruct[*entity.Entity](recordIds)
			}
		default:
			fmt.Println("Unknown key: " + key)
		}
		if !ok {
			return errors.New("field '" + key + "' could not be converted")
		}
	}

	return nil
}

as you can tell, it's horrible. All I want to do is:

• When certain fields are fetched,
• grab the models.RecordID(/s, multiples in an array)
• and grab the actual record instead and insert it right there.

Ideally, I would love to write something like

type MyData struct {
  ID models.recordID `json:"id"`
  OtherTable MyOtherType `json:"other_data"`
}

When unmarshalling, OtherTable would contain a models.RecordID- however, I want to intercept that, and specifically that, and instead construct my own struct in it's place.

Is that possible? Fetching nested data would be rather annoying otherwise...

Thank you and kind regards!

PS.: That's been like 4h of my day today... im so exhausted lol.

---

Edit: for context, here is SurrealDB's configuration with their custom tags https://github.com/surrealdb/surrealdb.go/blob/main/pkg/models/cbor.go

[fxamacker]

Ideally, I would love to write something like

type MyData struct {
 ID models.recordID `json:"id"`
 OtherTable MyOtherType `json:"other_data"`
}

When unmarshalling, OtherTable would contain a models.RecordID- however, I want to intercept that, and specifically that, and instead construct my own struct in it's place.

Hi @senpro-ingwersenk,

In general with fxamacker/cbor, you can interpret unmarshalling and unmarshalling by implementing Unmarshaler and Marshaler interfaces for custom data types, whether the data type is used as top-level or nested struct.

Maybe you can implement Unmarshaler for MyOtherType for your use case. Specifically, MyOtherType.UnmarshalCBOR() can decode models.RecordID and populate its fields as needed.

Something like this:

func (p *MyOtherType) UnmarshalCBOR(data []byte) error {
    // Decode raw data to models.RecordID.
    // Fetch other data, etc.
    // Populate MyOtherType struct fields.
}

PS.: That's been like 4h of my day today... im so exhausted lol.

I understand. I hope my comments are helpful despite being very general. 🙏 I'm not familiar with some of the internal data types in your code snippet and my current schedule doesn't allow me to dive into code bases unrelated to work (e.g. several projects/teams at work need my time and I haven't taken end-of-2024 break yet).

[IngwiePhoenix]

All good, thank you! I am sorry this response is coming in so late... But, basically, I ended up just rolling my own, lol.

I will revisit CBOR, some day - but for the time being, I'll stick to just JSON. :)