Apache Release Notes

Ambari 3.0.0.0-1

Security Improvements

User Interface Improvements

  • UI Framework Upgrades:

    • Upgraded jQuery and Bootstrap to latest versions (AMBARI-25289)
    • Fixed outdated ember-collection dependency (AMBARI-25988)
    • Fixed test failures caused by jQuery upgrade (AMBARI-26146)
  • UI Fixes and Enhancements:

  • Service UI Enhancements:

Core Improvements

Pinot 1.4.0

Apache Pinot 1.4.0 introduces major improvements over 1.3.0 across the query engine, ingestion, table management, security, and operational tooling.[1]

Multistage engine and queries

  • New Multistage Engine Lite Mode (beta) adds a scatter-gather style execution with per-leaf record limits, letting you run complex multi-stage queries (subqueries, window functions, etc.) safely at high QPS with low latency.[1]
  • A new physical optimizer (beta) for the multistage engine simplifies or removes redundant exchanges, supports group-by, joins, union-all, and can evaluate constant queries in the broker; it delivers large performance gains and lower CPU usage on colocated-join workloads.[1]
  • Enhancements include support for multiple window functions per plan, ASOF joins for time-aligned analytics, colocated joins with different partitioning, local replicated joins and exchanges to reduce shuffles, broadcast join hints, dynamic rule toggling in the optimizer, better type alias parsing, task throttling based on heap usage, and query cancellation with client-provided IDs.[1]

Ingestion, pauseless consumption, and logical tables

  • Pauseless consumption lets real-time ingestion continue while previous segments are built and uploaded, eliminating ingestion pauses and significantly reducing freshness gaps for real-time analytics; it includes validation, disaster recovery modes, observability metrics, and compatibility with upsert/dedup tables.[1]
  • Logical tables group multiple physical REALTIME and OFFLINE tables into a single logical table, behaving like a union-based view, simplifying Zookeeper scalability, ALTER TABLE workflows, topic/schema changes, table renames, and time-based layout changes while keeping them transparent to users.[1]
  • Time Series Engine moves into beta with UI support to visualize time-series query plans, a Prometheus-compatible query_range endpoint, and planner enhancements for limits, group limits, raw time handling, and metadata-driven planning, plus an end-to-end quickstart.[1]

Upsert, dedup, minion and indexing

  • Upsert and dedup are hardened by making segment creation time consistent via ZooKeeper metadata to avoid non-deterministic upsert decisions, introducing more flexible enablement controls, improving config handling, and deprecating old snapshot/preload flags in favor of new enums.[1]
  • Additional changes include allowing consumption during build for dedup/partial-upsert, better segment-creation-time tracking, stricter type validation for keys and time columns, improved error handling, and fixes for bad-state segments and TTL-related issues.[1]
  • Minion improvements focus on deterministic small-segment merges (using ZK creation times), the ability to skip dedup metadata for cold-tier segments, stronger validations, safer task selection, better observability, and more robust segment completion handling.[1]

New indexing, functions, plugins, and APIs

  • A multi-column text index allows a single text index across multiple columns with shared and per-column properties, cutting index overhead and speeding multi-field text search.[1]
  • Ingestion enhancements add JSON index heap caps, default Avro logical type support, real-time segment download fixes, a JSON Confluent Schema Registry decoder, and canonicalization of BigDecimal values for consistent comparison and dedup/upsert behavior.[1]
  • New scalar functions include JSON_MATCH extension points, JsonKeyValueArrayToMap, H3 gridDisk and gridDistance geospatial helpers, ISO 8601 date conversion, polymorphic ARRAY_LENGTH, CityHash, and additional date/time optimizer support such as a DATE_TRUNC optimizer.[1]
  • Plugin and API enhancements add an ArrowResponseEncoder for Arrow-format responses, S3 checksum support and MD5 behavior configuration, new gRPC query endpoints (including streaming), improved client timeout configurability, secret store interfaces, and additional utilities in Pinot tools.[1]Security, resource isolation, and operations

Security, resource isolation, and operations

  • Row-level security (RLS) is introduced to restrict row visibility per user or group, which is crucial for multi-tenant deployments; Groovy script static analysis adds a layer of safety for user scripts.[1]
  • Workload-based query resource isolation lets administrators define workload profiles with dedicated resource allocations, improving fairness and stability under mixed workloads.[1]
  • Operational changes include enforcing schemas for all tables, changing the default segment load mode to MMAP, better rebalance tooling (server-level batching, dry-run summaries, pre-checks, disk utilization info, minimize-data-movement options), more throttlers (segment download, reindex, star-tree rebuild), and richer metrics across multistage tasks, Zookeeper usage, index operations, message queues, netty memory, resource utilization, and consumer locks.[1]
  • Numerous UI enhancements (rebalance UI fixes, better query visualization, buttons for pause/resume ingestion and repair actions, improved table and schema forms, segment state filtering) and extensive bug fixes across ingestion, Kinesis integration, CLP, tests, and build tooling round out the 1.4.0 upgrade from 1.3.0.[1]

Sources

[1] https://docs.pinot.apache.org/basics/releases/1.4.0.md

Type to search, ESC to discard
Type to search, ESC to discard
Type to search, ESC to discard
  Last updated