Setting logging options in Sparkling Water

Once sparkContext is available please add the following:

  1. sc.setLogLevel("INFO")
  2. logging.getLogger().setLevel(logging.INFO)
  3. logging.info("My log")

 

The following does not work so don’t use it:

  1. log4jLogger = sc._jvm.org.apache.log4j
  2. LOGGER = log4jLogger.LogManager.getLogger(__name__)
Advertisement

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s