编程手册

# 开源动态

# 发布历史

Apache Flink: Apache Flink 1.8.0 Release Announcement (opens new window)

  1. 状态
    1. 连续增量的清理过期的 state 数据
    2. 状态恢复时的 schema 兼容性更强
    3. Savepoint 兼容性
  2. maven
    1. Flink 不再提供捆绑 Hadoop 包的发行版本
  3. Connectors (连接器)
    1. 增加 KafkaDeserializationSchema 能直接获取 kafka ConsumerRecord
    2. FlinkKafkaConsumer 将从恢复的数据中过滤不属于当前 topic 的 partition

Apache Flink: Apache Flink 1.9.0 Release Announcement (opens new window)

The Apache Flink project’s goal is to develop a stream processing system to unify and power many forms of real-time and offline data processing applications as well as event-driven applications. In this release, we have made a huge step forward in that effort, by integrating Flink’s stream and batch processing capabilities under a single, unified runtime.

  1. Fine-grained Batch Recovery (FLIP-1)

    1. Until Flink 1.9, task failures in batch jobs were recovered by canceling all tasks and restarting the whole job
    2. FLIP-1 : Fine Grained Recovery from Task Failures - Apache Flink - Apache Software Foundation (opens new window)

  3. State Processor API (FLIP-43)

  4. Stop-with-Savepoint (FLIP-34)

    1. bin/flink stop -p [:targetDirectory] :jobId

  5. Preview of the new Blink SQL Query Processor

    1. image

  6. Other Improvements to the Table API and SQL

    1. Scala-free Table API and SQL for Java users (FLIP-32)
    2. Rework of the Table API Type System (FLIP-37)

  7. Preview of Full Hive Integration (FLINK-10556)

    1. you will also be able to persist metadata of Flink tables in Metastore

# Feature

# Releases

Flink 1.12 Release 文档解读 - osc_zyjrm8ih的个人空间 - OSCHINA - 中文开源技术交流社区 (opens new window)

# Bugs

  • 描述

取消在基于yarn-cluster的群集上运行的作业，然后关闭群集时，不会删除推送网关上的指标*

  • 结论

通过调度程序来删除pushgateway上面的指标

In our product env, we also met the same problem. If pushgateway can implements TTL for pushed metrics[1], it'll very useful. But for now, we use a external schedule system to check whether the flink job is alive or not, then delete metrics by pushgateway's rest api[2].

[1]https://github.com/prometheus/pushgateway/issues/19

[2]https://github.com/prometheus/pushgateway#delete-method

← 状态原理 →