Logging with Sentry on Databricks

I couldn't find anything that laid this out really plainly, so I'm going to do so here. I'll update as I learn more and dig deeper, but this should give you the basics. It's super easy.

First, pip install the sentry-sdk library on your cluster (through the Databricks UI...not actually through a shell or anything)

Next, create a notebook and put this in there:

import logging

import sentry_sdk  
from sentry_sdk.integrations.logging import LoggingIntegration


logger = logging.getLogger('databricks')  
logger.setLevel(logging.DEBUG)

sentry_logging = LoggingIntegration(  
    level=logging.DEBUG,
    event_level=logging.DEBUG
)
sentry_sdk.init(  
  "your key here!",
  integrations=[sentry_logging]
)

Here is a link to the Sentry docs, which are pretty good. They'll give you more info about those kwargs, which have to do with breadcrumbs and which events are sent to sentry.

If you want, you can edit your cluster settings and add an init script as detailed here.

For some reason (I'm not sure if this is Python or Databricks), I'm not able to set the level of the root logger to INFO or anything, so that's why I had to create my own logger, which is probably good practice anyways. It was in Django, anyways. Unfortunately, using __name__ won't get you the name of your notebook - you'll just get __main__. So I named it myself. But you could also get the notebook name like described here.