Skip to content

Bump the processor-production-dependencies group across 1 directory with 15 updates #25

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 1 commit into from
Closed

Bump the processor-production-dependencies group across 1 directory with 15 updates #25

wants to merge 1 commit into from

Conversation

dependabot[bot]
Copy link
Contributor

@dependabot dependabot bot commented on behalf of github Nov 1, 2024 •
edited
Loading

Bumps the processor-production-dependencies group with 15 updates in the /processor directory:

Package From To
org.apache.commons:commons-lang3 3.14.0 3.17.0
org.apache.beam:beam-sdks-java-core 2.54.0 2.60.0
org.apache.beam:beam-runners-google-cloud-dataflow-java 2.54.0 2.60.0
org.apache.beam:beam-sdks-java-io-google-cloud-platform 2.54.0 2.60.0
org.apache.beam:beam-sdks-java-io-redis 2.54.0 2.60.0
com.fasterxml.jackson.datatype:jackson-datatype-joda 2.16.1 2.18.1
com.google.auto.value:auto-value 1.10.4 1.11.0
com.google.guava:guava 33.0.0-jre 33.3.1-jre
com.google.cloud:libraries-bom 26.33.0 26.50.0
org.apache.maven.surefire:surefire-junit47 3.2.5 3.5.1
org.apache.maven.plugins:maven-surefire-plugin 3.2.5 3.5.1
org.apache.maven.plugins:maven-compiler-plugin 3.12.1 3.13.0
org.apache.maven.plugins:maven-jar-plugin 3.3.0 3.4.2
org.apache.maven.plugins:maven-shade-plugin 3.5.2 3.6.0
org.codehaus.mojo:exec-maven-plugin 3.2.0 3.5.0

Updates org.apache.commons:commons-lang3 from 3.14.0 to 3.17.0

Updates org.apache.beam:beam-sdks-java-core from 2.54.0 to 2.60.0

Release notes

Sourced from org.apache.beam:beam-sdks-java-core's releases.

Beam 2.60.0 release

We are happy to present the new 2.60.0 release of Beam. This release includes both improvements and new functionality.

For more information on changes in 2.60.0, check out the detailed release notes.

Highlights

  • Added support for using vLLM in the RunInference transform (Python) (#32528)
  • [Managed Iceberg] Added support for streaming writes (#32451)
  • [Managed Iceberg] Added auto-sharding for streaming writes (#32612)
  • [Managed Iceberg] Added support for writing to dynamic destinations (#32565)

New Features / Improvements

  • Dataflow worker can install packages from Google Artifact Registry Python repositories (Python) (#32123).
  • Added support for Zstd codec in SerializableAvroCodecFactory (Java) (#32349)
  • Added support for using vLLM in the RunInference transform (Python) (#32528)
  • Prism release binaries and container bootloaders are now being built with the latest Go 1.23 patch. (#32575)
  • Prism
    • Prism now supports Bundle Finalization. (#32425)
  • Significantly improved performance of Kafka IO reads that enable commitOffsetsInFinalize by removing the data reshuffle from SDF implementation. (#31682).
  • Added support for dynamic writing in MqttIO (Java) (#19376)
  • Optimized Spark Runner parDo transform evaluator (Java) (#32537)
  • [Managed Iceberg] More efficient manifest file writes/commits (#32666)

Breaking Changes

  • In Python, assert_that now throws if it is not in a pipeline context instead of silently succeeding (#30771)
  • In Python and YAML, ReadFromJson now override the dtype from None to an explicit False. Most notably, string values like "123" are preserved as strings rather than silently coerced (and possibly truncated) to numeric values. To retain the old behavior, pass dtype=True (or any other value accepted by pandas.read_json).
  • Users of KafkaIO Read transform that enable commitOffsetsInFinalize might encounter pipeline graph compatibility issues when updating the pipeline. To mitigate, set the updateCompatibilityVersion option to the SDK version used for the original pipeline, example --updateCompatabilityVersion=2.58.1

Deprecations

  • Python 3.8 is reaching EOL and support is being removed in Beam 2.61.0. The 2.60.0 release will warn users when running on 3.8. (#31192)

Bugfixes

  • (Java) Fixed custom delimiter issues in TextIO (#32249, #32251).
  • (Java, Python, Go) Fixed PeriodicSequence backlog bytes reporting, which was preventing Dataflow Runner autoscaling from functioning properly (#32506).
  • (Java) Fix improper decoding of rows with schemas containing nullable fields when encoded with a schema with equal encoding positions but modified field order. (#32388).

Known Issues

N/A

... (truncated)

Changelog

Sourced from org.apache.beam:beam-sdks-java-core's changelog.

[2.60.0] - 2024-10-17

Highlights

  • Added support for using vLLM in the RunInference transform (Python) (#32528)
  • [Managed Iceberg] Added support for streaming writes (#32451)
  • [Managed Iceberg] Added auto-sharding for streaming writes (#32612)
  • [Managed Iceberg] Added support for writing to dynamic destinations (#32565)

I/Os

  • PubsubIO can validate that the Pub/Sub topic exists before running the Read/Write pipeline (Java) (#32465)

New Features / Improvements

  • Dataflow worker can install packages from Google Artifact Registry Python repositories (Python) (#32123).
  • Added support for Zstd codec in SerializableAvroCodecFactory (Java) (#32349)
  • Added support for using vLLM in the RunInference transform (Python) (#32528)
  • Prism release binaries and container bootloaders are now being built with the latest Go 1.23 patch. (#32575)
  • Prism
    • Prism now supports Bundle Finalization. (#32425)
  • Significantly improved performance of Kafka IO reads that enable commitOffsetsInFinalize by removing the data reshuffle from SDF implementation. (#31682).
  • Added support for dynamic writing in MqttIO (Java) (#19376)
  • Optimized Spark Runner parDo transform evaluator (Java) (#32537)
  • [Managed Iceberg] More efficient manifest file writes/commits (#32666)

Breaking Changes

  • In Python, assert_that now throws if it is not in a pipeline context instead of silently succeeding (#30771)
  • In Python and YAML, ReadFromJson now override the dtype from None to an explicit False. Most notably, string values like "123" are preserved as strings rather than silently coerced (and possibly truncated) to numeric values. To retain the old behavior, pass dtype=True (or any other value accepted by pandas.read_json).
  • Users of KafkaIO Read transform that enable commitOffsetsInFinalize might encounter pipeline graph compatibility issues when updating the pipeline. To mitigate, set the updateCompatibilityVersion option to the SDK version used for the original pipeline, example --updateCompatabilityVersion=2.58.1

Deprecations

  • Python 3.8 is reaching EOL and support is being removed in Beam 2.61.0. The 2.60.0 release will warn users when running on 3.8. (#31192)

Bugfixes

  • (Java) Fixed custom delimiter issues in TextIO (#32249, #32251).
  • (Java, Python, Go) Fixed PeriodicSequence backlog bytes reporting, which was preventing Dataflow Runner autoscaling from functioning properly (#32506).
  • (Java) Fix improper decoding of rows with schemas containing nullable fields when encoded with a schema with equal encoding positions but modified field order. (#32388).

[2.59.0] - 2024-09-11

Highlights

... (truncated)

Commits

Updates org.apache.beam:beam-runners-google-cloud-dataflow-java from 2.54.0 to 2.60.0

Release notes

Sourced from org.apache.beam:beam-runners-google-cloud-dataflow-java's releases.

Beam 2.60.0 release

We are happy to present the new 2.60.0 release of Beam. This release includes both improvements and new functionality.

For more information on changes in 2.60.0, check out the detailed release notes.

Highlights

  • Added support for using vLLM in the RunInference transform (Python) (#32528)
  • [Managed Iceberg] Added support for streaming writes (#32451)
  • [Managed Iceberg] Added auto-sharding for streaming writes (#32612)
  • [Managed Iceberg] Added support for writing to dynamic destinations (#32565)

New Features / Improvements

  • Dataflow worker can install packages from Google Artifact Registry Python repositories (Python) (#32123).
  • Added support for Zstd codec in SerializableAvroCodecFactory (Java) (#32349)
  • Added support for using vLLM in the RunInference transform (Python) (#32528)
  • Prism release binaries and container bootloaders are now being built with the latest Go 1.23 patch. (#32575)
  • Prism
    • Prism now supports Bundle Finalization. (#32425)
  • Significantly improved performance of Kafka IO reads that enable commitOffsetsInFinalize by removing the data reshuffle from SDF implementation. (#31682).
  • Added support for dynamic writing in MqttIO (Java) (#19376)
  • Optimized Spark Runner parDo transform evaluator (Java) (#32537)
  • [Managed Iceberg] More efficient manifest file writes/commits (#32666)

Breaking Changes

  • In Python, assert_that now throws if it is not in a pipeline context instead of silently succeeding (#30771)
  • In Python and YAML, ReadFromJson now override the dtype from None to an explicit False. Most notably, string values like "123" are preserved as strings rather than silently coerced (and possibly truncated) to numeric values. To retain the old behavior, pass dtype=True (or any other value accepted by pandas.read_json).
  • Users of KafkaIO Read transform that enable commitOffsetsInFinalize might encounter pipeline graph compatibility issues when updating the pipeline. To mitigate, set the updateCompatibilityVersion option to the SDK version used for the original pipeline, example --updateCompatabilityVersion=2.58.1

Deprecations

  • Python 3.8 is reaching EOL and support is being removed in Beam 2.61.0. The 2.60.0 release will warn users when running on 3.8. (#31192)

Bugfixes

  • (Java) Fixed custom delimiter issues in TextIO (#32249, #32251).
  • (Java, Python, Go) Fixed PeriodicSequence backlog bytes reporting, which was preventing Dataflow Runner autoscaling from functioning properly (#32506).
  • (Java) Fix improper decoding of rows with schemas containing nullable fields when encoded with a schema with equal encoding positions but modified field order. (#32388).

Known Issues

N/A

... (truncated)

Changelog

Sourced from org.apache.beam:beam-runners-google-cloud-dataflow-java's changelog.

[2.60.0] - 2024-10-17

Highlights

  • Added support for using vLLM in the RunInference transform (Python) (#32528)
  • [Managed Iceberg] Added support for streaming writes (#32451)
  • [Managed Iceberg] Added auto-sharding for streaming writes (#32612)
  • [Managed Iceberg] Added support for writing to dynamic destinations (#32565)

I/Os

  • PubsubIO can validate that the Pub/Sub topic exists before running the Read/Write pipeline (Java) (#32465)

New Features / Improvements

  • Dataflow worker can install packages from Google Artifact Registry Python repositories (Python) (#32123).
  • Added support for Zstd codec in SerializableAvroCodecFactory (Java) (#32349)
  • Added support for using vLLM in the RunInference transform (Python) (#32528)
  • Prism release binaries and container bootloaders are now being built with the latest Go 1.23 patch. (#32575)
  • Prism
    • Prism now supports Bundle Finalization. (#32425)
  • Significantly improved performance of Kafka IO reads that enable commitOffsetsInFinalize by removing the data reshuffle from SDF implementation. (#31682).
  • Added support for dynamic writing in MqttIO (Java) (#19376)
  • Optimized Spark Runner parDo transform evaluator (Java) (#32537)
  • [Managed Iceberg] More efficient manifest file writes/commits (#32666)

Breaking Changes

  • In Python, assert_that now throws if it is not in a pipeline context instead of silently succeeding (#30771)
  • In Python and YAML, ReadFromJson now override the dtype from None to an explicit False. Most notably, string values like "123" are preserved as strings rather than silently coerced (and possibly truncated) to numeric values. To retain the old behavior, pass dtype=True (or any other value accepted by pandas.read_json).
  • Users of KafkaIO Read transform that enable commitOffsetsInFinalize might encounter pipeline graph compatibility issues when updating the pipeline. To mitigate, set the updateCompatibilityVersion option to the SDK version used for the original pipeline, example --updateCompatabilityVersion=2.58.1

Deprecations

  • Python 3.8 is reaching EOL and support is being removed in Beam 2.61.0. The 2.60.0 release will warn users when running on 3.8. (#31192)

Bugfixes

  • (Java) Fixed custom delimiter issues in TextIO (#32249, #32251).
  • (Java, Python, Go) Fixed PeriodicSequence backlog bytes reporting, which was preventing Dataflow Runner autoscaling from functioning properly (#32506).
  • (Java) Fix improper decoding of rows with schemas containing nullable fields when encoded with a schema with equal encoding positions but modified field order. (#32388).

[2.59.0] - 2024-09-11

Highlights

... (truncated)

Commits

Updates org.apache.beam:beam-sdks-java-io-google-cloud-platform from 2.54.0 to 2.60.0

Release notes

Sourced from org.apache.beam:beam-sdks-java-io-google-cloud-platform's releases.

Beam 2.60.0 release

We are happy to present the new 2.60.0 release of Beam. This release includes both improvements and new functionality.

For more information on changes in 2.60.0, check out the detailed release notes.

Highlights

  • Added support for using vLLM in the RunInference transform (Python) (#32528)
  • [Managed Iceberg] Added support for streaming writes (#32451)
  • [Managed Iceberg] Added auto-sharding for streaming writes (#32612)
  • [Managed Iceberg] Added support for writing to dynamic destinations (#32565)

New Features / Improvements

  • Dataflow worker can install packages from Google Artifact Registry Python repositories (Python) (#32123).
  • Added support for Zstd codec in SerializableAvroCodecFactory (Java) (#32349)
  • Added support for using vLLM in the RunInference transform (Python) (#32528)
  • Prism release binaries and container bootloaders are now being built with the latest Go 1.23 patch. (#32575)
  • Prism
    • Prism now supports Bundle Finalization. (#32425)
  • Significantly improved performance of Kafka IO reads that enable commitOffsetsInFinalize by removing the data reshuffle from SDF implementation. (#31682).
  • Added support for dynamic writing in MqttIO (Java) (#19376)
  • Optimized Spark Runner parDo transform evaluator (Java) (#32537)
  • [Managed Iceberg] More efficient manifest file writes/commits (#32666)

Breaking Changes

  • In Python, assert_that now throws if it is not in a pipeline context instead of silently succeeding (#30771)
  • In Python and YAML, ReadFromJson now override the dtype from None to an explicit False. Most notably, string values like "123" are preserved as strings rather than silently coerced (and possibly truncated) to numeric values. To retain the old behavior, pass dtype=True (or any other value accepted by pandas.read_json).
  • Users of KafkaIO Read transform that enable commitOffsetsInFinalize might encounter pipeline graph compatibility issues when updating the pipeline. To mitigate, set the updateCompatibilityVersion option to the SDK version used for the original pipeline, example --updateCompatabilityVersion=2.58.1

Deprecations

  • Python 3.8 is reaching EOL and support is being removed in Beam 2.61.0. The 2.60.0 release will warn users when running on 3.8. (#31192)

Bugfixes

  • (Java) Fixed custom delimiter issues in TextIO (#32249, #32251).
  • (Java, Python, Go) Fixed PeriodicSequence backlog bytes reporting, which was preventing Dataflow Runner autoscaling from functioning properly (#32506).
  • (Java) Fix improper decoding of rows with schemas containing nullable fields when encoded with a schema with equal encoding positions but modified field order. (#32388).

Known Issues

N/A

... (truncated)

Changelog

Sourced from org.apache.beam:beam-sdks-java-io-google-cloud-platform's changelog.

[2.60.0] - 2024-10-17

Highlights

  • Added support for using vLLM in the RunInference transform (Python) (#32528)
  • [Managed Iceberg] Added support for streaming writes (#32451)
  • [Managed Iceberg] Added auto-sharding for streaming writes (#32612)
  • [Managed Iceberg] Added support for writing to dynamic destinations (#32565)

I/Os

  • PubsubIO can validate that the Pub/Sub topic exists before running the Read/Write pipeline (Java) (#32465)

New Features / Improvements

  • Dataflow worker can install packages from Google Artifact Registry Python repositories (Python) (#32123).
  • Added support for Zstd codec in SerializableAvroCodecFactory (Java) (#32349)
  • Added support for using vLLM in the RunInference transform (Python) (#32528)
  • Prism release binaries and container bootloaders are now being built with the latest Go 1.23 patch. (#32575)
  • Prism
    • Prism now supports Bundle Finalization. (#32425)
  • Significantly improved performance of Kafka IO reads that enable commitOffsetsInFinalize by removing the data reshuffle from SDF implementation. (#31682).
  • Added support for dynamic writing in MqttIO (Java) (#19376)
  • Optimized Spark Runner parDo transform evaluator (Java) (#32537)
  • [Managed Iceberg] More efficient manifest file writes/commits (#32666)

Breaking Changes

  • In Python, assert_that now throws if it is not in a pipeline context instead of silently succeeding (#30771)
  • In Python and YAML, ReadFromJson now override the dtype from None to an explicit False. Most notably, string values like "123" are preserved as strings rather than silently coerced (and possibly truncated) to numeric values. To retain the old behavior, pass dtype=True (or any other value accepted by pandas.read_json).
  • Users of KafkaIO Read transform that enable commitOffsetsInFinalize might encounter pipeline graph compatibility issues when updating the pipeline. To mitigate, set the updateCompatibilityVersion option to the SDK version used for the original pipeline, example --updateCompatabilityVersion=2.58.1

Deprecations

  • Python 3.8 is reaching EOL and support is being removed in Beam 2.61.0. The 2.60.0 release will warn users when running on 3.8. (#31192)

Bugfixes

  • (Java) Fixed custom delimiter issues in TextIO (#32249, #32251).
  • (Java, Python, Go) Fixed PeriodicSequence backlog bytes reporting, which was preventing Dataflow Runner autoscaling from functioning properly (#32506).
  • (Java) Fix improper decoding of rows with schemas containing nullable fields when encoded with a schema with equal encoding positions but modified field order. (#32388).

[2.59.0] - 2024-09-11

Highlights

... (truncated)

Commits

Updates org.apache.beam:beam-sdks-java-io-redis from 2.54.0 to 2.60.0

Release notes

Sourced from org.apache.beam:beam-sdks-java-io-redis's releases.

Beam 2.60.0 release

We are happy to present the new 2.60.0 release of Beam. This release includes both improvements and new functionality.

For more information on changes in 2.60.0, check out the detailed release notes.

Highlights

  • Added support for using vLLM in the RunInference transform (Python) (#32528)
  • [Managed Iceberg] Added support for streaming writes (#32451)
  • [Managed Iceberg] Added auto-sharding for streaming writes (#32612)
  • [Managed Iceberg] Added support for writing to dynamic destinations (#32565)

New Features / Improvements

  • Dataflow worker can install packages from Google Artifact Registry Python repositories (Python) (#32123).
  • Added support for Zstd codec in SerializableAvroCodecFactory (Java) (#32349)
  • Added support for using vLLM in the RunInference transform (Python) (#32528)
  • Prism release binaries and container bootloaders are now being built with the latest Go 1.23 patch. (#32575)
  • Prism
    • Prism now supports Bundle Finalization. (#32425)
  • Significantly improved performance of Kafka IO reads that enable commitOffsetsInFinalize by removing the data reshuffle from SDF implementation. (#31682).
  • Added support for dynamic writing in MqttIO (Java) (#19376)
  • Optimized Spark Runner parDo transform evaluator (Java) (#32537)
  • [Managed Iceberg] More efficient manifest file writes/commits (#32666)

Breaking Changes

  • In Python, assert_that now throws if it is not in a pipeline context instead of silently succeeding (#30771)
  • In Python and YAML, ReadFromJson now override the dtype from None to an explicit False. Most notably, string values like "123" are preserved as strings rather than silently coerced (and possibly truncated) to numeric values. To retain the old behavior, pass dtype=True (or any other value accepted by pandas.read_json).
  • Users of KafkaIO Read transform that enable commitOffsetsInFinalize might encounter pipeline graph compatibility issues when updating the pipeline. To mitigate, set the updateCompatibilityVersion option to the SDK version used for the original pipeline, example --updateCompatabilityVersion=2.58.1

Deprecations

  • Python 3.8 is reaching EOL and support is being removed in Beam 2.61.0. The 2.60.0 release will warn users when running on 3.8. (#31192)

Bugfixes

  • (Java) Fixed custom delimiter issues in TextIO (#32249, #32251).
  • (Java, Python, Go) Fixed PeriodicSequence backlog bytes reporting, which was preventing Dataflow Runner autoscaling from functioning properly (#32506).
  • (Java) Fix improper decoding of rows with schemas containing nullable fields when encoded with a schema with equal encoding positions but modified field order. (#32388).

Known Issues

N/A

... (truncated)

Changelog

Sourced from org.apache.beam:beam-sdks-java-io-redis's changelog.

[2.60.0] - 2024-10-17

Highlights

  • Added support for using vLLM in the RunInference transform (Python) (#32528)
  • [Managed Iceberg] Added support for streaming writes (#32451)
  • [Managed Iceberg] Added auto-sharding for streaming writes (#32612)
  • [Managed Iceberg] Added support for writing to dynamic destinations (#32565)

I/Os

  • PubsubIO can validate that the Pub/Sub topic exists before running the Read/Write pipeline (Java) (#32465)

New Features / Improvements

  • Dataflow worker can install packages from Google Artifact Registry Python repositories (Python) (#32123).
  • Added support for Zstd codec in SerializableAvroCodecFactory (Java) (#32349)
  • Added support for using vLLM in the RunInference transform (Python) (#32528)
  • Prism release binaries and container bootloaders are now being built with the latest Go 1.23 patch. (#32575)
  • Prism
    • Prism now supports Bundle Finalization. (#32425)
  • Significantly improved performance of Kafka IO reads that enable commitOffsetsInFinalize by removing the data reshuffle from SDF implementation. (#31682).
  • Added support for dynamic writing in MqttIO (Java) (#19376)
  • Optimized Spark Runner parDo transform evaluator (Java) (#32537)
  • [Managed Iceberg] More efficient manifest file writes/commits (#32666)

Breaking Changes

  • In Python, assert_that now throws if it is not in a pipeline context instead of silently succeeding (#30771)
  • In Python and YAML, ReadFromJson now override the dtype from None to an explicit False. Most notably, string values like "123" are preserved as strings rather than silently coerced (and possibly truncated) to numeric values. To retain the old behavior, pass dtype=True (or any other value accepted by pandas.read_json).
  • Users of KafkaIO Read transform that enable commitOffsetsInFinalize might encounter pipeline graph compatibility issues when updating the pipeline. To mitigate, set the updateCompatibilityVersion option to the SDK version used for the original pipeline, example --updateCompatabilityVersion=2.58.1

Deprecations

  • Python 3.8 is reaching EOL and support is being removed in Beam 2.61.0. The 2.60.0 release will warn users when running on 3.8. (#31192)

Bugfixes

  • (Java) Fixed custom delimiter issues in TextIO (#32249, #32251).
  • (Java, Python, Go) Fixed PeriodicSequence backlog bytes reporting, which was preventing Dataflow Runner autoscaling from functioning properly (#32506).
  • (Java) Fix improper decoding of rows with schemas containing nullable fields when encoded with a schema with equal encoding positions but modified field order. (#32388).

[2.59.0] - 2024-09-11

Highlights

... (truncated)

Commits

Updates org.apache.beam:beam-runners-google-cloud-dataflow-java from 2.54.0 to 2.60.0

Release notes

Sourced from org.apache.beam:beam-runners-google-cloud-dataflow-java's releases.

Beam 2.60.0 release

We are happy to present the new 2.60.0 release of Beam. This release includes both improvements and new functionality.

For more information on changes in 2.60.0, check out the detailed release notes.

Highlights

  • Added support for using vLLM in the RunInference transform (Python) (#32528)
  • [Managed Iceberg] Added support for streaming writes (#32451)
  • [Managed Iceberg] Added auto-sharding for streaming writes (#32612)
  • [Managed Iceberg] Added support for writing to dynamic destinations (#32565)

New Features / Improvements

  • Dataflow worker can install packages from Google Artifact Registry Python repositories (Python) (#32123).
  • Added support for Zstd codec in SerializableAvroCodecFactory (Java) (#32349)
  • Added support for using vLLM in the RunInference transform (Python) (#32528)
  • Prism release binaries and container bootloaders are now being built with the latest Go 1.23 patch. (#32575)
  • Prism
    • Prism now supports Bundle Finalization. (#32425)
  • Significantly improved performance of Kafka IO reads that enable commitOffsetsInFinalize by removing the data reshuffle from SDF implementation. (#31682).
  • Added support for dynamic writing in MqttIO (Java) (#19376)
  • Optimized Spark Runner parDo transform evaluator (Java) (#32537)
  • [Managed Iceberg] More efficient manifest file writes/commits (#32666)

Breaking Changes

  • In Python, assert_that now throws if it is not in a pipeline context instead of silently succeeding (#30771)
  • In Python and YAML, ReadFromJson now override the dtype from None to an explicit False. Most notably, string values like "123" are preserved as strings rather than silently coerced (and possibly truncated) to numeric values. To retain the old behavior, pass dtype=True (or any other value accepted by pandas.read_json).
  • Users of KafkaIO Read transform that enable commitOffsetsInFinalize might encounter pipeline graph compatibility issues when updating the pipeline. To mitigate, set the updateCompatibilityVersion option to the SDK version used for the original pipeline, example --updateCompatabilityVersion=2.58.1

Deprecations

  • Python 3.8 is reaching EOL and support is being removed in Beam 2.61.0. The 2.60.0 release will warn users when running on 3.8. (#31192)

Bugfixes

  • (Java) Fixed custom delimiter issues in TextIO (#32249, #32251).
  • (Java, Python, Go) Fixed PeriodicSequence backlog bytes reporting, which was preventing Dataflow Runner autoscaling from functioning properly (#32506).
  • (Java) Fix improper decoding of rows with schemas containing nullable fields when encoded with a schema with equal encoding positions but modified field order. (#32388).

Known Issues

N/A

... (truncated)

Changelog

Sourced from org.apache.beam:beam-runners-google-cloud-dataflow-java's changelog.

[2.60.0] - 2024-10-17

Highlights

  • Added support for using vLLM in the RunInference transform (Python) (#32528)
  • [Managed Iceberg] Added support for streaming writes (#32451)
  • [Managed Iceberg] Added auto-sharding for streaming writes (#32612)
  • [Managed Iceberg] Added support for writing to dynamic destinations (#32565)

I/Os

  • PubsubIO can validate that the Pub/Sub topic exists before running the Read/Write pipeline (Java) (#32465)

New Features / Improvements

  • Dataflow worker can install packages from Google Artifact Registry Python repositories (Python) (#32123).
  • Added support for Zstd codec in SerializableAvroCodecFactory (Java) (#32349)
  • Added support for using vLLM in the RunInference transform (Python) (#32528)
  • Prism release binaries and container bootloaders are now being built with the latest Go 1.23 patch. (#32575)
  • Prism
    • Prism now supports Bundle Finalization. (#32425)
  • Significantly improved performance of Kafka IO reads that enable commitOffsetsInFinalize by removing the data reshuffle from SDF implementation. (#31682).
  • Added support for dynamic writing in MqttIO (Java) (#19376)
  • Optimized Spark Runner parDo transform evaluator (Java) (#32537)
  • [Managed Iceberg] More efficient manifest file writes/commits (#32666)

Breaking Changes

  • In Python, assert_that now throws if it is not in a pipeline context instead of silently succeeding (#30771)
  • In Python and YAML, ReadFromJson now override the dtype from None to an explicit False. Most notably, string values like "123" are preserved as strings rather than silently coerced (and possibly truncated) to numeric values. To retain the old behavior, pass dtype=True (or any other value accepted by pandas.read_json).
  • Users of KafkaIO Read transform that enable commitOffsetsInFinalize might encounter pipeline graph compatibility issues when updating the pipeline. To mitigate, set the updateCompatibilityVersion option to the SDK version used for the original pipeline, example --updateCompatabilityVersion=2.58.1

Deprecations

  • Python 3.8 is reaching EOL and support is being removed in Beam 2.61.0. The 2.60.0 release will warn users when running on 3.8. (#31192)

Bugfixes

  • (Java) Fixed custom delimiter issues in TextIO (#32249, #32251).
  • (Java, Python, Go) Fixed PeriodicSequence backlog bytes reporting, which was preventing Dataflow Runner autoscaling from functioning properly (#32506).
  • (Java) Fix improper decoding of rows with schemas containing nullable fields when encoded with a schema with equal encoding positions but modified field order. (#32388).

[2.59.0] - 2024-09-11

Highlights

... (truncated)

Commits

@dependabot
Bump the processor-production-dependencies group across 1 directory w…
4f2e1c4 
…ith 15 updates

Bumps the processor-production-dependencies group with 15 updates in the /processor directory:

| Package | From | To |
| --- | --- | --- |
| org.apache.commons:commons-lang3 | `3.14.0` | `3.17.0` |
| [org.apache.beam:beam-sdks-java-core](https://github.com/apache/beam) | `2.54.0` | `2.60.0` |
| [org.apache.beam:beam-runners-google-cloud-dataflow-java](https://github.com/apache/beam) | `2.54.0` | `2.60.0` |
| [org.apache.beam:beam-sdks-java-io-google-cloud-platform](https://github.com/apache/beam) | `2.54.0` | `2.60.0` |
| [org.apache.beam:beam-sdks-java-io-redis](https://github.com/apache/beam) | `2.54.0` | `2.60.0` |
| [com.fasterxml.jackson.datatype:jackson-datatype-joda](https://github.com/FasterXML/jackson-datatype-joda) | `2.16.1` | `2.18.1` |
| [com.google.auto.value:auto-value](https://github.com/google/auto) | `1.10.4` | `1.11.0` |
| [com.google.guava:guava](https://github.com/google/guava) | `33.0.0-jre` | `33.3.1-jre` |
| [com.google.cloud:libraries-bom](https://github.com/googleapis/java-cloud-bom) | `26.33.0` | `26.50.0` |
| org.apache.maven.surefire:surefire-junit47 | `3.2.5` | `3.5.1` |
| [org.apache.maven.plugins:maven-surefire-plugin](https://github.com/apache/maven-surefire) | `3.2.5` | `3.5.1` |
| [org.apache.maven.plugins:maven-compiler-plugin](https://github.com/apache/maven-compiler-plugin) | `3.12.1` | `3.13.0` |
| [org.apache.maven.plugins:maven-jar-plugin](https://github.com/apache/maven-jar-plugin) | `3.3.0` | `3.4.2` |
| [org.apache.maven.plugins:maven-shade-plugin](https://github.com/apache/maven-shade-plugin) | `3.5.2` | `3.6.0` |
| [org.codehaus.mojo:exec-maven-plugin](https://github.com/mojohaus/exec-maven-plugin) | `3.2.0` | `3.5.0` |



Updates `org.apache.commons:commons-lang3` from 3.14.0 to 3.17.0

Updates `org.apache.beam:beam-sdks-java-core` from 2.54.0 to 2.60.0
- [Release notes](https://github.com/apache/beam/releases)
- [Changelog](https://github.com/apache/beam/blob/master/CHANGES.md)
- [Commits](apache/beam@v2.54.0...v2.60.0)

Updates `org.apache.beam:beam-runners-google-cloud-dataflow-java` from 2.54.0 to 2.60.0
- [Release notes](https://github.com/apache/beam/releases)
- [Changelog](https://github.com/apache/beam/blob/master/CHANGES.md)
- [Commits](apache/beam@v2.54.0...v2.60.0)

Updates `org.apache.beam:beam-sdks-java-io-google-cloud-platform` from 2.54.0 to 2.60.0
- [Release notes](https://github.com/apache/beam/releases)
- [Changelog](https://github.com/apache/beam/blob/master/CHANGES.md)
- [Commits](apache/beam@v2.54.0...v2.60.0)

Updates `org.apache.beam:beam-sdks-java-io-redis` from 2.54.0 to 2.60.0
- [Release notes](https://github.com/apache/beam/releases)
- [Changelog](https://github.com/apache/beam/blob/master/CHANGES.md)
- [Commits](apache/beam@v2.54.0...v2.60.0)

Updates `org.apache.beam:beam-runners-google-cloud-dataflow-java` from 2.54.0 to 2.60.0
- [Release notes](https://github.com/apache/beam/releases)
- [Changelog](https://github.com/apache/beam/blob/master/CHANGES.md)
- [Commits](apache/beam@v2.54.0...v2.60.0)

Updates `org.apache.beam:beam-sdks-java-io-google-cloud-platform` from 2.54.0 to 2.60.0
- [Release notes](https://github.com/apache/beam/releases)
- [Changelog](https://github.com/apache/beam/blob/master/CHANGES.md)
- [Commits](apache/beam@v2.54.0...v2.60.0)

Updates `org.apache.beam:beam-sdks-java-io-redis` from 2.54.0 to 2.60.0
- [Release notes](https://github.com/apache/beam/releases)
- [Changelog](https://github.com/apache/beam/blob/master/CHANGES.md)
- [Commits](apache/beam@v2.54.0...v2.60.0)

Updates `com.fasterxml.jackson.datatype:jackson-datatype-joda` from 2.16.1 to 2.18.1
- [Commits](FasterXML/jackson-datatype-joda@jackson-datatype-joda-2.16.1...jackson-datatype-joda-2.18.1)

Updates `com.google.auto.value:auto-value` from 1.10.4 to 1.11.0
- [Release notes](https://github.com/google/auto/releases)
- [Commits](google/auto@auto-value-1.10.4...auto-value-1.11.0)

Updates `com.google.guava:guava` from 33.0.0-jre to 33.3.1-jre
- [Release notes](https://github.com/google/guava/releases)
- [Commits](https://github.com/google/guava/commits)

Updates `com.google.cloud:libraries-bom` from 26.33.0 to 26.50.0
- [Release notes](https://github.com/googleapis/java-cloud-bom/releases)
- [Changelog](https://github.com/googleapis/java-cloud-bom/blob/main/release-please-config.json)
- [Commits](googleapis/java-cloud-bom@v26.33.0...v26.50.0)

Updates `org.apache.maven.surefire:surefire-junit47` from 3.2.5 to 3.5.1

Updates `org.apache.maven.plugins:maven-surefire-plugin` from 3.2.5 to 3.5.1
- [Release notes](https://github.com/apache/maven-surefire/releases)
- [Commits](apache/maven-surefire@surefire-3.2.5...surefire-3.5.1)

Updates `org.apache.maven.plugins:maven-compiler-plugin` from 3.12.1 to 3.13.0
- [Release notes](https://github.com/apache/maven-compiler-plugin/releases)
- [Commits](apache/maven-compiler-plugin@maven-compiler-plugin-3.12.1...maven-compiler-plugin-3.13.0)

Updates `org.apache.maven.plugins:maven-surefire-plugin` from 3.2.5 to 3.5.1
- [Release notes](https://github.com/apache/maven-surefire/releases)
- [Commits](apache/maven-surefire@surefire-3.2.5...surefire-3.5.1)

Updates `org.apache.maven.plugins:maven-jar-plugin` from 3.3.0 to 3.4.2
- [Release notes](https://github.com/apache/maven-jar-plugin/releases)
- [Commits](apache/maven-jar-plugin@maven-jar-plugin-3.3.0...maven-jar-plugin-3.4.2)

Updates `org.apache.maven.plugins:maven-shade-plugin` from 3.5.2 to 3.6.0
- [Release notes](https://github.com/apache/maven-shade-plugin/releases)
- [Commits](apache/maven-shade-plugin@maven-shade-plugin-3.5.2...maven-shade-plugin-3.6.0)

Updates `org.codehaus.mojo:exec-maven-plugin` from 3.2.0 to 3.5.0
- [Release notes](https://github.com/mojohaus/exec-maven-plugin/releases)
- [Commits](mojohaus/exec-maven-plugin@3.2.0...3.5.0)

---
updated-dependencies:
- dependency-name: org.apache.commons:commons-lang3
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: processor-production-dependencies
- dependency-name: org.apache.beam:beam-sdks-java-core
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: processor-production-dependencies
- dependency-name: org.apache.beam:beam-runners-google-cloud-dataflow-java
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: processor-production-dependencies
- dependency-name: org.apache.beam:beam-sdks-java-io-google-cloud-platform
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: processor-production-dependencies
- dependency-name: org.apache.beam:beam-sdks-java-io-redis
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: processor-production-dependencies
- dependency-name: org.apache.beam:beam-runners-google-cloud-dataflow-java
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: processor-production-dependencies
- dependency-name: org.apache.beam:beam-sdks-java-io-google-cloud-platform
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: processor-production-dependencies
- dependency-name: org.apache.beam:beam-sdks-java-io-redis
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: processor-production-dependencies
- dependency-name: com.fasterxml.jackson.datatype:jackson-datatype-joda
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: processor-production-dependencies
- dependency-name: com.google.auto.value:auto-value
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: processor-production-dependencies
- dependency-name: com.google.guava:guava
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: processor-production-dependencies
- dependency-name: com.google.cloud:libraries-bom
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: processor-production-dependencies
- dependency-name: org.apache.maven.surefire:surefire-junit47
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: processor-production-dependencies
- dependency-name: org.apache.maven.plugins:maven-surefire-plugin
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: processor-production-dependencies
- dependency-name: org.apache.maven.plugins:maven-compiler-plugin
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: processor-production-dependencies
- dependency-name: org.apache.maven.plugins:maven-surefire-plugin
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: processor-production-dependencies
- dependency-name: org.apache.maven.plugins:maven-jar-plugin
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: processor-production-dependencies
- dependency-name: org.apache.maven.plugins:maven-shade-plugin
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: processor-production-dependencies
- dependency-name: org.codehaus.mojo:exec-maven-plugin
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: processor-production-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
@dependabot dependabot bot added the dependencies Pull requests that update a dependency file label Nov 1, 2024
@dependabot dependabot bot mentioned this pull request Nov 1, 2024
Closed
@dependabot Dependabot
Copy link
Contributor Author

dependabot bot commented on behalf of github Dec 1, 2024

Superseded by #27.

@dependabot dependabot bot closed this Dec 1, 2024
@dependabot dependabot bot deleted the dependabot/maven/processor/processor-production-dependencies-9eac07ba6b branch December 1, 2024 03:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

dependencies Pull requests that update a dependency file

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

0 participants