接着上一篇,将mysql的数据导入kafka中
public static void main(String[] arg) throws Exception {
TypeInformation[] fieldTypes = new TypeInformation[] { BasicTypeInfo.STRING_TYPE_INFO };
RowTypeInfo rowTypeInfo = new RowTypeInfo(fieldTypes);
JDBCInputFormat jdbcInputFormat = JDBCInputFormat.buildJDBCInputFormat().setDrivername("com.mysql.jdbc.Driver")
.setDBUrl("jdbc:mysql://*:3306/tablename?characterEncoding=utf8")
.setUsername("*").setPassword("*")
.setQuery("select LOGIC_CODE from *").setRowTypeInfo(rowTypeInfo).finish();
final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
DataSource<Row> s = env.createInput(jdbcInputFormat);
BatchTableEnvironment tableEnv = new BatchTableEnvironment(env, TableConfig.DEFAULT());
tableEnv.registerDataSet("t2", s);
Table query = tableEnv.sqlQuery("select * from t2");
DataSet<String> result = tableEnv.toDataSet(query, Row.class).map(new MapFunction<Row, String>(){
@Override
public String map(Row value) throws Exception {
return value.toString() ;
}
});
logger.info("read db end");
KafkaOutputFormat kafkaOutput = KafkaOutputFormat.buildKafkaOutputFormat()
.setBootstrapServers("*:9092").setTopic("search_test_whk").setAcks("all").setBatchSize("1000")
.setBufferMemory("100000").setLingerMs("1").setRetries("2").finish();
result.output(kafkaOutput);
logger.info("write kafka end");
env.execute("Flink add data source");
}
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 [email protected] 举报,一经查实,本站将立刻删除。