Open
Description
Language models use very well the context, and the column name can give useful context. For instance the value "mouse" should probably be embedded differently if the column name is "peripheral" versus "animal".
My suggestion is to have an option to embed entries with "{column_name}: {entry}" to enable exploiting this piece of context.
Of course, ideally we should benchmark this option, to see how / when it brings value.
The benefit would be to disambiguate "State: New York" and "City: New York"
Activity