SendGrid 'substitutions' must be strings

[PlusAsh]

I've been working with the GlitchTip project to identify and resolve an issue affecting my deployment of the GlitchTip platform and my ability to send alert emails through Sendgrid. The Glitchtip platform uses django-anymail to send emails.

In debugging my issue, we identified that Sendgrid requires 'personalisation' and 'substitution' values to be passed as strings, and we have been able to resolve my issue with Glitchtip by casting the specific value passed in from Glitchtip to a string.

However, it is my opinion this fix belongs in anymail (and other middlewares/backends that talk to SendGrid) rather than in the consuming software - that fix being any substitution of personalisation data passed in should be casted to a string before being sent to the SendGrid API.

I recognise of course this is an opinion and welcome your feedback and contribution to our issue!

You can see a draft merge request for the Glitchtip software here: https://gitlab.com/glitchtip/glitchtip-backend/-/merge_requests/1378

[medmunds]

Thanks for the report. Couple of thoughts/questions:

• Unless your unique_id needs to be rendered in template content, it might be more appropriate to use Anymail's merge_metadata rather than merge_data. Anymail specifically handles exactly the problem you describe for metadata.
• For SendGrid's usual dynamic templates, the personalizations dynamic_template_data can be any JSON structure—arbitrarily nested dicts and lists. So I don't think there's any reasonable way for Anymail to process that to convert numbers to strings. (Does dynamic_template_data really not work with numbers?)
• If you're using SendGrid's legacy templates (or legacy merge fields directly in your message body), I think there is some Anymail code we could improve. It looks like SendGrid's legacy substitutions must have string values. We could update this code to coerce numbers to strings and report any more-complex type as unsupported.

[PlusAsh]

Thanks @medmunds, if Anymail handles our issue when using merge_metadata we'll work on a swap to that instead. Thanks!

[bufke]

Thanks for looking into this everyone. I think my confusion with the original glitchtip implementation stems from here. merge_data is mentioned heavily but it sounds like merge_metadata is the best, unless you want to do template substitution (we don't).

[medmunds]

merge_metadata is relatively new (compared to merge_data), and its detailed docs ended up in a different section just below its non-merge counterpart. I think it could be helpful to also highlight it in the Batch sending section you'd been looking at.

I am going to treat this as a bug in Anymail's construction of SendGrid "substitutions" from merge_metadata (when using a legacy template, or no template). Anymail's docs say that integers should be OK. And SendGrid's error message is this exceptionally unhelpful HTTP 400 response:

{
 "errors": [
   {
     "message": "Bad Request",
     "field": null,
     "help": null
   }
 ]
}