分类
devops

flink和spark的类装载器

Table of Contents

flink

classloader.resolve-order: child-first # parent-first
#classloader.parent-first-patterns.default
classloader.parent-first-patterns.additional: com.codahale.metrics
java.lang.LinkageError: loader constraint violation: when resolving method 'void org.apache.flink.dropwizard.metrics.DropwizardHistogramWrapper.<init>(com.codahale.metrics.Histogram)' the class loader org.apache.flink.util.ChildFirstClassLoader @1f4f7e8c of the current class, org/apache/iceberg/flink/sink/IcebergStreamWriterMetrics, and the class loader 'app' for the method's defining class, org/apache/flink/dropwizard/metrics/DropwizardHistogramWrapper, have different Class objects for the type com/codahale/metrics/Histogram used in the signature (org.apache.iceberg.flink.sink.IcebergStreamWriterMetrics is in unnamed module of loader org.apache.flink.util.ChildFirstClassLoader @1f4f7e8c, parent loader 'app'; org.apache.flink.dropwizard.metrics.DropwizardHistogramWrapper is in unnamed module of loader 'app')
    at org.apache.iceberg.flink.sink.IcebergStreamWriterMetrics.<init>(IcebergStreamWriterMetrics.java:54)
    at org.apache.iceberg.flink.sink.IcebergStreamWriter.open(IcebergStreamWriter.java:56)
    at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.initializeStateAndOpenOperators(RegularOperatorChain.java:107)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreStateAndGates(StreamTask.java:858)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$restoreInternal$5(StreamTask.java:812)
    at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.call(StreamTaskActionExecutor.java:100)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreInternal(StreamTask.java:812)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask.java:771)
    at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:970)
    at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:939)
    at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:763)
    at org.apache.flink.runtime.taskmanager.Task.run(Task.java:575)
    at java.base/java.lang.Thread.run(Unknown Source)

spark

--conf "spark.driver.extraClassPath="
--conf "spark.executor.extraClassPath="