postgres-cdc-flink:使用Debezium流式传输PostgreSQL CDC,并使用Flink使用状态流进行充实-源码

上传者: 42099302 | 上传时间: 2021-08-25 17:35:15 | 文件大小: 67KB | 文件类型: ZIP
使用Flink丰富Kafka流与另一个流 环境设定 安装PostgreSQL 11+ 设置PostgreSQL以允许使用pgoutput将Debezium转换为CDC。 参考 在您的机器/集群上设置Apache Kafka(使用Kafka Connect) 从安装Debezium PostgreSQL连接器 运行Apache Kafka和Kafka Connect 在PostgreSQL中创建表transactions和customers (SQL文件) 使用请求主体向您的Kafka Connect REST接口创建POST请求,如下所示 { " name " : " postgres_cdc " , " config " : { " connector.class " : " io.debezium.connector.postgresql.PostgresCon

文件下载

资源详情

[{"title":"( 45 个子文件 67KB ) postgres-cdc-flink:使用Debezium流式传输PostgreSQL CDC,并使用Flink使用状态流进行充实-源码","children":[{"title":"postgres-cdc-flink-main","children":[{"title":".gitignore <span style='color:#111;'> 4.40KB </span>","children":null,"spread":false},{"title":".project <span style='color:#111;'> 939B </span>","children":null,"spread":false},{"title":"README.md <span style='color:#111;'> 1.94KB </span>","children":null,"spread":false},{"title":"pom.xml <span style='color:#111;'> 7.49KB </span>","children":null,"spread":false},{"title":".github","children":[{"title":"workflows","children":[{"title":"maven.yml <span style='color:#111;'> 715B </span>","children":null,"spread":false}],"spread":true}],"spread":true},{"title":".vscode","children":[{"title":"launch.json <span style='color:#111;'> 826B </span>","children":null,"spread":false},{"title":"settings.json <span style='color:#111;'> 2B </span>","children":null,"spread":false}],"spread":true},{"title":"LICENSE <span style='color:#111;'> 11.10KB </span>","children":null,"spread":false},{"title":"sql","children":[{"title":"tables.sql <span style='color:#111;'> 1.90KB </span>","children":null,"spread":false}],"spread":true},{"title":"src","children":[{"title":"test","children":[{"title":"java","children":[{"title":"dev","children":[{"title":"ramottamado","children":[{"title":"java","children":[{"title":"flink","children":[{"title":"TransactionsEnrichmentStreamingJobIntegrationTest.java <span style='color:#111;'> 5.60KB </span>","children":null,"spread":false},{"title":"functions","children":[{"title":"DestinationAccountKeySelectorTest.java <span style='color:#111;'> 1.78KB </span>","children":null,"spread":false},{"title":"helper","children":[{"title":"TestSourceFunction.java <span style='color:#111;'> 1.71KB </span>","children":null,"spread":false},{"title":"TestTimestampAssigner.java <span style='color:#111;'> 1.25KB </span>","children":null,"spread":false}],"spread":false},{"title":"EnrichTransactionsWithCustomersJoinFunctionTest.java <span style='color:#111;'> 4.53KB </span>","children":null,"spread":false},{"title":"EnrichEnrichedTransactionsWithCustomersJoinFunctionTest.java <span style='color:#111;'> 5.46KB </span>","children":null,"spread":false},{"title":"EnrichedTransactionsToStringMapFunctionTest.java <span style='color:#111;'> 1.84KB </span>","children":null,"spread":false},{"title":"EnvelopeParserMapFunctionTest.java <span style='color:#111;'> 2.62KB </span>","children":null,"spread":false}],"spread":false},{"title":"util","children":[{"title":"jackson","children":[{"title":"helper","children":[{"title":"ClassWithCustomSerDe.java <span style='color:#111;'> 2.52KB </span>","children":null,"spread":false}],"spread":false},{"title":"deserializer","children":[{"title":"MicroTimestampDeserializerTest.java <span style='color:#111;'> 3.15KB </span>","children":null,"spread":false}],"spread":false},{"title":"serializer","children":[{"title":"TimestampSerializerTest.java <span style='color:#111;'> 2.60KB </span>","children":null,"spread":false}],"spread":false}],"spread":false},{"title":"serialization","children":[{"title":"DebeziumJSONEnvelopeDeserializationSchemaTest.java <span style='color:#111;'> 4.70KB </span>","children":null,"spread":false},{"title":"JSONValueDeserializationSchemaTest.java <span style='color:#111;'> 3.46KB </span>","children":null,"spread":false}],"spread":false}],"spread":false}],"spread":false}],"spread":false}],"spread":true}],"spread":true}],"spread":true}],"spread":true},{"title":"main","children":[{"title":"resources","children":[{"title":"log4j2.properties <span style='color:#111;'> 1.01KB </span>","children":null,"spread":false}],"spread":true},{"title":"java","children":[{"title":"dev","children":[{"title":"ramottamado","children":[{"title":"java","children":[{"title":"flink","children":[{"title":"functions","children":[{"title":"EnrichTransactionsWithCustomersJoinFunction.java <span style='color:#111;'> 4.84KB </span>","children":null,"spread":false},{"title":"DestinationAccountKeySelector.java <span style='color:#111;'> 1.46KB </span>","children":null,"spread":false},{"title":"EnrichEnrichedTransactionsWithCustomersJoinFunction.java <span style='color:#111;'> 3.91KB </span>","children":null,"spread":false},{"title":"EnrichedTransactionsToStringMapFunction.java <span style='color:#111;'> 1.69KB </span>","children":null,"spread":false},{"title":"EnvelopeParserMapFunction.java <span style='color:#111;'> 2.06KB </span>","children":null,"spread":false}],"spread":false},{"title":"api","children":[{"title":"schema","children":[{"title":"ClassWithTimestamp.java <span style='color:#111;'> 1.09KB </span>","children":null,"spread":false}],"spread":false}],"spread":false},{"title":"KafkaTransactionsEnrichmentStreamingJob.java <span style='color:#111;'> 6.77KB </span>","children":null,"spread":false},{"title":"config","children":[{"title":"ParameterConfig.java <span style='color:#111;'> 1.69KB </span>","children":null,"spread":false}],"spread":false},{"title":"schema","children":[{"title":"TransactionsWithTimestamp.java <span style='color:#111;'> 1.79KB </span>","children":null,"spread":false},{"title":"EnrichedTransactions.java <span style='color:#111;'> 5.04KB </span>","children":null,"spread":false},{"title":"EnrichedTransactionsWithTimestamp.java <span style='color:#111;'> 1.86KB </span>","children":null,"spread":false},{"title":"Transactions.java <span style='color:#111;'> 3.80KB </span>","children":null,"spread":false},{"title":"Customers.java <span style='color:#111;'> 3.33KB </span>","children":null,"spread":false}],"spread":false},{"title":"util","children":[{"title":"ParameterUtils.java <span style='color:#111;'> 1.88KB </span>","children":null,"spread":false},{"title":"jackson","children":[{"title":"deserializer","children":[{"title":"MicroTimestampDeserializer.java <span style='color:#111;'> 2.06KB </span>","children":null,"spread":false}],"spread":false},{"title":"serializer","children":[{"title":"TimestampSerializer.java <span style='color:#111;'> 2.25KB </span>","children":null,"spread":false}],"spread":false}],"spread":false},{"title":"kafka","children":[{"title":"KafkaProperties.java <span style='color:#111;'> 2.02KB </span>","children":null,"spread":false}],"spread":false},{"title":"serialization","children":[{"title":"DebeziumJSONEnvelopeDeserializationSchema.java <span style='color:#111;'> 2.31KB </span>","children":null,"spread":false},{"title":"EnrichedTransactionsKafkaSerializationSchema.java <span style='color:#111;'> 2.40KB </span>","children":null,"spread":false},{"title":"JSONValueDeserializationSchema.java <span style='color:#111;'> 2.03KB </span>","children":null,"spread":false}],"spread":false}],"spread":false},{"title":"AbstractTransactionsEnrichmentStreamingJob.java <span style='color:#111;'> 4.61KB </span>","children":null,"spread":false}],"spread":false}],"spread":false}],"spread":false}],"spread":false}],"spread":true}],"spread":true}],"spread":true},{"title":".classpath <span style='color:#111;'> 10.75KB </span>","children":null,"spread":false}],"spread":true}],"spread":true}]

评论信息

免责申明

【只为小站】的资源来自网友分享,仅供学习研究,请务必在下载后24小时内给予删除,不得用于其他任何用途,否则后果自负。基于互联网的特殊性,【只为小站】 无法对用户传输的作品、信息、内容的权属或合法性、合规性、真实性、科学性、完整权、有效性等进行实质审查;无论 【只为小站】 经营者是否已进行审查,用户均应自行承担因其传输的作品、信息、内容而可能或已经产生的侵权或权属纠纷等法律责任。
本站所有资源不代表本站的观点或立场,基于网友分享,根据中国法律《信息网络传播权保护条例》第二十二条之规定,若资源存在侵权或相关问题请联系本站客服人员,zhiweidada#qq.com,请把#换成@,本站将给予最大的支持与配合,做到及时反馈和处理。关于更多版权及免责申明参见 版权及免责申明