Known Limitations Iceberg

The following limitations apply to Apache Iceberg in ODP 3.3.6.5:

• Spark 4.1.1 requires the Spark 4.1 Iceberg runtime: Use iceberg-spark-runtime-4.1_2.13. Using the Spark 3.5 Iceberg runtime with Spark 4.1.1 can cause an AbstractMethodError when the Iceberg extensions are loaded. Some earlier Mpack builds might include the incorrect Spark 3.5 runtime. For resolution steps, see Resolve AbstractMethodError on Spark 4.1.1.

• SparkSessionCatalog requires fully qualified names for metadata tables: When you use SparkSessionCatalog, specify the complete catalog, database, table, and metadata table name. For example, use spark_catalog.default.sales.snapshots instead of sales.snapshots. For more information, see Resolve "Table or view not found" for Metadata Tables.

• Spark 3.3 on JDK 8 supports Iceberg 1.8.1 or earlier: Iceberg 1.9.0 and later require JDK 11-compatible bytecode. If Spark 3.3 runs on JDK 8, use Iceberg 1.8.1 or an earlier compatible version.

• Some Spark installations include an older Iceberg version in the POM: The released POM files for Spark 3.3.3 and Spark 3.5.1 specify iceberg.version=1.4.3. Until an updated Mpack provides the newer runtime, use the Iceberg runtime recommended in Choose the Iceberg Runtime for Your Spark Version.