-
Notifications
You must be signed in to change notification settings - Fork 1
feat(observability): add structured logging #51
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
| abstract readonly statusCode: number; | ||
|
|
||
| /** Whether this error type is generally safe to retry */ | ||
| abstract readonly isRetryable: boolean; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see here we are using isRetryable, but then in the wide-event file we have this
setError(error: Error): this {
const isAppKitError = "code" in error && "statusCode" in error;
const errorCause = (error as any).cause;
this.data.error = {
type: error.name,
code: isAppKitError ? (error as any).code : "UNKNOWN_ERROR",
message: error.message,
retriable: isAppKitError ? (error as any).retriable : false,
cause: errorCause ? String(errorCause) : undefined,
};
return this;
}shouldn't it be
retriable: isAppKitError ? (error as any).isRetryable : false,|
|
||
| // Check path prefixes | ||
| for (const prefix of EXCLUDED_PATH_PREFIXES) { | ||
| if (cleanPath.startsWith(prefix) || cleanPath.includes(prefix)) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this could just be
| if (cleanPath.startsWith(prefix) || cleanPath.includes(prefix)) { | |
| if (cleanPath.includes(prefix)) { |
no?
| * | ||
| * @example | ||
| * ```typescript | ||
| * import { ValidationError, AuthenticationError } from "@databricks/appkit"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if this is what we want to do, we are missing all the error exports in the root index file 😄
| } | ||
|
|
||
| // set in AsyncLocalStorage so downstream code can access via logger.event() | ||
| eventStorage.enterWith(wideEvent); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
as talked in the call, let's make sure that we need to use enterWith instead of run 😄
| type: error.name, | ||
| code: isAppKitError ? (error as any).code : "UNKNOWN_ERROR", | ||
| message: error.message, | ||
| retriable: isAppKitError ? (error as any).retriable : false, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| retriable: isAppKitError ? (error as any).retriable : false, | |
| retriable: isAppKitError ? (error as any).isRetryable : false, |
pkosiec
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice job overall! A few comments from my side
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
General error comment: I think I mentioned that before, but I'm not a fan of centralizing e.g. errors which are used just once in a given package or scoped directory.
If they are used in multiple places (e.g. across different plugins), then sure, having them in a single central place makes sense. But for the package-specific ones, I'd rather keep them close to the usage (e.g. server plugin errors within the server dir) and just use the AppKitError abstract class.
It's not a merge blocker, just an opinion, perhaps it's worth to discuss our approach within the team.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Quick question - have you researched existing logging solutions? There may be an already existing option that meets our requirements and would eliminate the need for separate maintenance 🤔
| resource: this.createResource(config), | ||
| autoDetectResources: false, | ||
| sampler: new AlwaysOnSampler(), | ||
| sampler: new AppKitSampler(), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you please test a deployed app with OpenTelemetry enabled in Apps runtime? I think it would be great to double-check everything works flawlessly there 👍
From my observations sampling result might differ between local dev and runtime, that's why the AlwaysOnSampler was set before (same as previous default OpenTelemetry config).
Summary
WideEventclass for collecting rich request metadata (method, path, user, component, execution, stream, error context)AsyncLocalStoragefor automatic context propagation across interceptors and nested callsDetails
Standardized Errors
New error hierarchy with
AppKitErrorbase class providing:code,statusCode,isRetryable)ValidationError,AuthenticationError,ConfigurationError,ConnectionError,ExecutionError,InitializationError,ServerError,TunnelErrortoJSON()serializationValidationError.missingField("warehouseId"))Structured Logging
createLogger(scope)returns a logger with:logger.info(req, "Processing %s", id)tracks logs in WideEventlogger.event(req?)returns the current WideEvent (from AsyncLocalStorage or explicit req)Wide-Event Tracking (Honeycomb-style)
Request-scoped
WideEventcollects:Sampling
Configurable via
APPKIT_SAMPLE_RATEenv var with smart defaults: