如何解决在Scala中跳过CSV文件的第一行
我想读写一个csv文件而忽略第一行，因为标题是从第二行开始的。
val df =df1.withColumn("index",monotonicallyIncreasingId()).filter(col("index") > 1).drop("index")
这不能解决我的问题。
解决方法
可能是左防加入吗？
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width,initial-scale=1.0">
<title>Manage Fees</title>
<!-- ADDING FONT AWESOME CSS -->
<link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css">
<!-- ADDING BOOTSTRAP CSS -->
<link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.5.0/css/bootstrap.min.css"
integrity="sha384-9aIt2nRpC12Uk9gS9baDl411NQApFmC26EwAOH8WgZl5MYYxFfc+NcPb1dKGj7Sk" crossorigin="anonymous">
<!-- ADDING STYLE.CSS -->
<link rel="stylesheet" href="/css/style.css">
</head>
<body onload="showfees()">
<!-- ADDING BUTTONS LIKE SHOW MATCHES,ADD MATCHES USING CSS BOOTSTRAP -->
<!-- <button type="button" class="btn btn-secondary showfees" onclick="showfees()">Show Fees</button> -->
<button type="button" class="btn btn-secondary addfees" onclick="addfees()">Add Fees</button>
<button type="button" class="btn btn-danger showfees" onclick="showfees()">Back</button>
<!-- ADDING SEARCH BAR -->
<input type="text" class="searchbox" id="searchbox" onkeyup="mySearch()" placeholder="Search by ID...">
<!-- CREATING REGISTRATION PAGE FOR ADDING FEES -->
<table class="feesregistration" id="feesregistration">
<!-- <tr>
<td>
<label>ID :</label>
</td>
<td>
<input type="text" class="feesid" id="feesid">
</td>
</tr> -->
<tr>
<td>
<label>Name :</label>
</td>
<td>
<input type="text" class="playerNameFees" id="playerNameFees" maxlength="40">
</td>
<td>
<label class="playerNameFeeserror"></label>
</td>
</tr>
<tr>
<td>
<label>Fees For :</label>
</td>
<td>
<select class="feesFor" id="feesFor">
<option value="select">---Select---</option>
<option value="court">Court</option>
<option value="tournament">Tournament</option>
<option value="both">Both</option>
</select>
</td>
<td>
<label class="feesForerror"></label>
</td>
</tr>
<tr>
<td>
<label>Fee Type :</label>
</td>
<td>
<select class="feesType" id="feesType">
<option value="select">---Select---</option>
<option value="monthly">Monthly</option>
<option value="halfyearly">Half Yearly</option>
<option value="annually">Annually</option>
</select>
</td>
<td>
<label class="feesTypeerror"></label>
</td>
</tr>
<tr>
<td>
<label>Date :</label>
</td>
<td>
<input type="date" class="feesdate">
</td>
<td>
<label class="feesdateerror"></label>
</td>
</tr>
<tr>
<td>
<button type="button" class="btn btn-success savefees" onclick="saveFees()">SAVE</button>
</td>
<td>
<button type="button" class="btn btn-info clearfees" onclick="clearfees()">CLEAR</button>
</td>
</tr>
</table>
<!-- ADDING BOOTSTRAP JS -->
<!-- JS,Popper.js,and jQuery -->
<script src="https://code.jquery.com/jquery-3.5.1.slim.min.js"
integrity="sha384-DfXdz2htPH0lsSSs5nCTpuj/zy4C+OGpamoFVy38MVBnE+IbbVYUew+OrCXaRkfj"
crossorigin="anonymous"></script>
<script src="https://cdn.jsdelivr.net/npm/popper.js@1.16.0/dist/umd/popper.min.js"
integrity="sha384-Q6E9RHvbIyZFJoft+2mJbHaEWldlvI9IOYy5n3zV9zzTtmI3UksdQRVvoxMfooAo"
crossorigin="anonymous"></script>
<script src="https://stackpath.bootstrapcdn.com/bootstrap/4.5.0/js/bootstrap.min.js"
integrity="sha384-OgVRvuATP1z7JjHLkuOU7Xw704+h835Lr+6QL9UvYjZE3Ipu6Tp75j7Bh/kR0JKI"
crossorigin="anonymous"></script>
<!-- ADDING INDEX.JS -->
<script src="/js/sidebar.js"></script>
<script src="/js/index.js"></script>
</body>
</html>
或使用rdd过滤器。
header = df.limit(1)
df.join(header,df.columns,'left_anti').show()
让我告诉您我的尝试。 如果标头出现在csv文件的第二行中，我们需要忽略第一行。
val df=spark.read.text("/yourFilePath/my.csv").withColumn("row_id",monotonically_increasing_id)
val cols=df.select("value").filter('row_id===1).first.mkString.split(",")
val df2 = df.filter('row_id>1).
withColumn("temp",split(col("value"),",")).
select((0 until cols.length).map(i => col("temp").getItem(i).as(cols.apply(i))): _*)
之前
+------------+------+
| value|row_id|
+------------+------+
| aasadasd| 0|
|name,age,des| 1|
| a,2,dd| 2|
| b,5,ff| 3|
+------------+------+
之后
+----+---+---+
|name|age|des|
+----+---+---+
|a |2 |dd |
|b |5 |ff |
+----+---+---+
版权声明：本文内容由互联网用户自发贡献，该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容， 请发送邮件至 dio@foxmail.com 举报，一经查实，本站将立刻删除。