[references] Recognition - Allow built-in datasets usage

[sarjil77]

hey @felixdittrich92,

now we are able to use the builtin dataset, and it is working fine.

[sarjil77]

@felixdittrich92, here i am able to use the builtin dataset but also facing one issue:

getting this type of error in some of the dataset, it is workign fine for the SVNH but for others which have space between words are facing this kind of issue.

ValueError: some characters cannot be found in 'vocab'.                          Please check the input string ACUTE CADCOVASCULAR and the vocabulary 0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ!"#$%&'()*+,-./:;<=>?@[\]^_`{|}~°£€¥¢฿àâéèêëîïôùûüçÀÂÉÈÊËÎÏÔÙÛÜÇ

and the error is because of input_string: ACUTE CADCOVASCULAR, so space between them causes the error and when i tried to add the space in the vocab.

i got error like regarding:
the vocabulary size in your model does not match the pre-trained checkpoint.

please have a look on this. :)

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 96.55%. Comparing base (18b8db9) to head (f4e9d0a).
Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1904      +/-   ##
==========================================
- Coverage   96.65%   96.55%   -0.11%     
==========================================
  Files         166      166              
  Lines        7991     8000       +9     
==========================================
  Hits         7724     7724              
- Misses        267      276       +9     

Flag	Coverage Δ
unittests	96.55% <100.00%> (-0.11%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[felixdittrich92]

@felixdittrich92, here i am able to use the builtin dataset but also facing one issue:

getting this type of error in some of the dataset, it is workign fine for the SVNH but for others which have space between words are facing this kind of issue.

ValueError: some characters cannot be found in 'vocab'.                          Please check the input string ACUTE CADCOVASCULAR and the vocabulary 0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ!"#$%&'()*+,-./:;<=>?@[\]^_`{|}~°£€¥¢฿àâéèêëîïôùûüçÀÂÉÈÊËÎÏÔÙÛÜÇ

and the error is because of input_string: ACUTE CADCOVASCULAR, so space between them causes the error and when i tried to add the space in the vocab.

i got error like regarding: the vocabulary size in your model does not match the pre-trained checkpoint.

please have a look on this. :)

Thanks I will have a look on monday 👍🏼
I think we should filter the possible choices here because some datasets like SROIE contains partial word group annotations (including whitespaces) instead of single word annotations.

[sarjil77]

yes, that sounds good,
and the issue is only becuase of partial word group annotations (including whitespaces).

OR we can include the whitespace in the vocab.

let me know for other further changes required here.

[felixdittrich92]

As discussed some updates required :)

[felixdittrich92]

Related to: #1830

[felixdittrich92]

One last thing 👍

[felixdittrich92]

Thanks 🤗 Looks good now 👍