Proposal: Replace YAML rule definitions with Go DSL for hook instrumentation

[pdelewski]

This proposal suggests replacing the current YAML-based instrumentation rule definitions with a Go-based internal DSL. This would provide greater flexibility, type safety, and enable advanced use cases that are difficult or impossible to express in YAML.

Motivation

The current YAML approach works well for simple declarative scenarios:

  server_hook:
    target: net/http
    func: ServeHTTP
    recv: serverHandler
    before: BeforeServeHTTP
    after: AfterServeHTTP
    path: "github.com/.../nethttp/server"

However, YAML becomes limiting when more sophisticated rewriting logic is required:

• No programmatic control: Cannot express conditional logic or dynamic transformations
• Limited AST manipulation: The raw field allows code injection but cannot modify function signatures, rename parameters, or perform context-sensitive transformations
• No type safety: YAML is stringly-typed, errors are only caught at runtime
• Duplication: Similar rules require copy-paste with minor variations

Proposed Solution

Define instrumentation rules as plain Go code using an internal DSL:

  package nethttp_hooks

  import "github.com/open-telemetry/opentelemetry-go-compile-instrumentation/hooks"

  func ProvideHooks() []*hooks.Hook {
      return []*hooks.Hook{
          {
              Target: hooks.InjectTarget{
                  Package:  "net/http",
                  Function: "ServeHTTP",
                  Receiver: "serverHandler",
              },
              Hooks: &hooks.InjectFunctions{
                  Before: "BeforeServeHTTP",
                  After:  "AfterServeHTTP",
                  From:   "github.com/.../nethttp/server",
              },
          },
      }
  }

For advanced cases like runtime instrumentation, the Go DSL enables full AST manipulation:

  func (r *RuntimeHookProvider) ProvideHooks() []*hooks.Hook {
      return []*hooks.Hook{
          {
              Target: hooks.InjectTarget{
                  Package:  "runtime",
                  Function: "newproc1",
              },
              Rewrite: func(node ast.Node) (ast.Node, error) {
                  funcDecl := node.(*ast.FuncDecl)
                  // Rename unnamed return values
                  renameReturnValues(funcDecl)
                  // Inject context propagation
                  stmts, _ := parseSnippet(`defer func(){...}()`)
                  funcDecl.Body.List = append(stmts, funcDecl.Body.List...)
                  return funcDecl, nil
              },
          },
      }
  }

  func (r *RuntimeHookProvider) GetStructModifications() []hooks.StructModification {
      return []hooks.StructModification{
          {
              Package:    "runtime",
              StructName: "g",
              AddFields: []hooks.StructField{
                  {Name: "otel_trace_context", Type: "interface{}"},
                  {Name: "otel_baggage_container", Type: "interface{}"},
              },
          },
      }
  }

Benefits

Aspect	YAML	Go DSL
Type safety	Runtime errors	Compile-time checks
Expressiveness	Limited to predefined fields	Full language power
AST manipulation	Only raw code injection	Complete control via go/ast
Conditional logic	Not supported	Native Go conditionals
Code reuse	Copy-paste	Functions, interfaces, composition
IDE support	Basic YAML highlighting	Full Go tooling (autocomplete, refactoring)
Testing	Difficult	Standard Go tests

Backwards Compatibility

YAML support could be maintained as a convenience layer that generates Go DSL internally. Simple rules would continue to work in YAML, while advanced use cases would use Go directly.

Reference Implementation

A working implementation of this approach exists in https://github.com/pdelewski/go-build-interceptor, demonstrating:

• Before/After hooks
• Function rewriting with full AST access
• Struct modification
• File generation

Next Steps

1. Define the hooks package API
2. Implement hook provider parsing/loading
3. Add YAML-to-Go DSL converter for migration
4. Update existing instrumentations to use Go DSL

[y1yang0]

+1 for this proposal.

I have a couple of questions

1. Regarding the roadmap, is this planned for the 1.0.0 release?

2. Regarding the implementation, after switching to this Go DSL model, my understanding is that the tool would still need to read the nethttp_hooks.go file and parse its source code as text to extract the hook information. It seems difficult for the tool to actually execute this Go code to obtain the Hook objects directly. Is my understanding correct?

[kakkoyun]

I want to be explicit: I’m opposed to replacing our YAML configuration with a Go-based configuration format.

Why:
1. User experience > developer convenience. These configs will primarily be authored and maintained by platform engineers and operators. They expect a configuration language with familiar workflows (diffs, reviews, schema validation, IDE hints), not code.
2. “Yet another DSL” tax is real. A Go-based format increases onboarding cost, creates a harder boundary between “who can edit config” and “who can write Go,” and tends to accumulate complexity over time (dependency management, compilation/runtime behavior, version pinning, execution context, security considerations). (I know LLMs could be helpful with this case).
3. Reviewability and safety. Config should be inert data. Code-as-config raises questions like: what can it import, what can it execute, what are the side effects, how do we sandbox it, and how do we make it deterministic? Those are solvable problems, but they’re also expensive problems we don’t need if YAML is sufficient.
4. Most configs are minimal. The common case is small and declarative. Optimizing for the rare “complex” use case by making everything code is the wrong tradeoff.

Middle ground that preserves UX while addressing the concerns:
• Make the specification declarative and struct-based in Go (a stable API/AST, not arbitrary code).
• Use struct tags to support YAML (and JSON if desired).
• Provide:
• a validator/linter (CI + local),
• JSON Schema generation for autocomplete + editor validation,
• good error messages and examples.

If we need custom logic:
• Support a reference-based extension mechanism: YAML points to external Go components that live alongside the config (plain Go files with build tags, or a plugin/module approach), instead of turning the config itself into Go. (Which we already have)

This gives us strong typing and validation, keeps configs accessible and reviewable, and avoids turning the configuration layer into a mini application platform.