如何解决AWS athena + Glue初始查询问题
尝试使用Athena和Glue对存储在S3中的JSON文件创建第一个查询时,出现以下错误:
Your query has the following error(s):
SYNTAX_ERROR: line 1:8: SELECT * not allowed in queries without FROM clause
This query ran against the "peds" database,unless qualified by the query. Please post the error message on our forum or contact customer support with Query Id: a0f8c091-5738-423e-ac0f-7c4a6002273b.
我的查询是控制台单击预览表时由控制台自动生成的测试查询,它具有FROM子句。这是查询:
SELECT * FROM "peds"."data" limit 10
我已经尝试了以下方法:
- 我打开了我的存储桶权限以确保它不是安全问题
- 在线发现名称中的连字符存在问题,并确保我没有这个问题
- 确保Athena写入的存储桶与我的Athena服务位于同一区域。
有什么想法我做错了吗?
**编辑** 样本json(实际json较大,但应该具有相似的架构):
{ "PatentBulkData" : [ {"patentCaseMetadata":{"applicationNumberText":{"value":"04558530","electronicText":"04558530"},"filingDate":"1956-01-11","applicationTypeCategory":"Utility","partyBag":{"applicantBagOrInventorBagOrOwnerBag":[{"primaryExaminerOrAssistantExaminerOrAuthorizedOfficer":[{"name":{"personNameOrOrganizationNameOrEntityName":[{"personFullName":"SCHOFER,JOSEPH"}]}}]},{"partyIdentifierOrContact":[{"name":{"personNameOrOrganizationNameOrEntityName":[{"personStructuredName":{"lastName":"FRENCH,HUGHES & DOESCHER"}}]},"postalAddressBag":{"postalAddress":[{"postalStructuredAddress":{"addressLineText":[{"value":"SUITE 1107"},{"value":"1825 K STREET,N. W."}],"cityName":"WASHINGTON","geographicRegionName":[{"value":"DC"}],"countryCode":"US","postalCode":"20006"}}]}}]}]},"groupArtUnitNumber":{"value":"1505","electronicText":"1505"},"applicationConfirmationNumber":"9850","patentClassificationBag":{"cpcClassificationBagOrIPCClassificationOrECLAClassificationBag":[{"ipOfficeCode":"US","mainNationalClassification":{"nationalClass":"526","nationalSubclass":"351000"}}]},"businessEntityStatusCategory":"UNDISCOUNTED","firstInventorToFileIndicator":"Other","inventionTitle":{"content":["SOLID POLYMERS OF OLEFINS AND PRODUCTION OF SUCH POLYMERS"]},"applicationStatusCategory":"Patented Case","applicationStatusDate":"1983-01-12","officialFileLocationCategory":"FILE REPOSITORY (FRANCONIA)","officialFileLocationDate":"2019-12-09","patentGrantIdentification":{"patentNumber":"4376851","grantDate":"1983-03-15"}},"prosecutionHistoryDataBag":{"prosecutionHistoryData":[{"eventDate":"2001-09-21","eventCode":"SETS","eventDescriptionText":"Set Application Status"},{"eventDate":"2000-03-01","eventCode":"FOND","eventDescriptionText":"Case Found"},"eventCode":"LOST","eventDescriptionText":"Case Reported Lost"},{"eventDate":"1987-02-20","eventCode":"TMOS","eventDescriptionText":"Termination of Official Search"},{"eventDate":"1987-01-12","eventCode":"SRCH","eventDescriptionText":"Official Search Conducted"},{"eventDate":"1986-12-19",{"eventDate":"1986-12-18",{"eventDate":"1986-11-07",{"eventDate":"1986-11-04",{"eventDate":"1986-06-20",{"eventDate":"1986-05-01",{"eventDate":"1986-04-23",{"eventDate":"1985-10-01",{"eventDate":"1985-09-24",{"eventDate":"1985-06-20",{"eventDate":"1982-11-24","eventCode":"N084","eventDescriptionText":"Issue Fee Payment Verified"},{"eventDate":"1982-11-19","eventCode":"MN/=.","eventDescriptionText":"Mail Notice of Allowance"},"eventCode":"N/=.","eventDescriptionText":"Notice of Allowance Data Verification Completed"},{"eventDate":"1982-04-30","eventCode":"MCTNF","eventDescriptionText":"Mail Non-Final Rejection"},"eventCode":"CTNF","eventDescriptionText":"Non-Final Rejection"},{"eventDate":"1982-11-18","eventCode":"DOCK","eventDescriptionText":"Case Docketed to Examiner in GAU"}]},"assignmentDataBag":{"assignmentData":[{"reelNumber":"13158","frameNumber":"318","documentReceivedDate":"2002-08-09","recordedDate":"2002-08-05","mailDate":"2002-10-10","pageTotalQuantity":4,"conveyanceText":"RELEASE OF SECURITY AGREEMENT","assignorBag":{"assignor":[{"executionDate":"2002-04-30","contactOrPublicationContact":[{"name":{"personNameOrOrganizationNameOrEntityName":[{"value":"BANKAMERICA BUSINESS CREDIT,INC.,AS AGENT"}]}}]}]},"assigneeBag":{"assignee":[{"contactOrPublicationContact":[{"name":{"personNameOrOrganizationNameOrEntityName":[{"value":"HYUNDAI MOTOR AMERICA"}]},"postalAddressBag":{"postalAddress":[{"postalAddressText":[{"sequenceNumber":"1","value":"10550 TALBERT AVENUE"},{"sequenceNumber":"2","value":"FOUNTAIN VALLEY CALIFORNIA 92708"}]}]}}]}]},"correspondenceAddress":{"partyIdentifierOrContact":[{"name":{"personNameOrOrganizationNameOrEntityName":[{"value":"JENKENS & GILCHRIST,P.C."}]},"value":"ANDRE M. SZUWALSKI"},"value":"1445 ROSS AVENUE,STE. 3200"},{"sequenceNumber":"3","value":"DALLAS,TX 75202-2799"}]}]}}]},"sequenceNumber":"1"}],"assignmentTotalQuantity":1},"st96Version":"V3_1","ipoVersion":"US_V8_0"}
] }
解决方法
我将您提供给S3的示例JSON上载了,并运行了一个爬网程序,该爬虫为我提供了以下定义。
CREATE EXTERNAL TABLE `test`(
`patentbulkdata` array<struct<patentcasemetadata:struct<applicationnumbertext:struct<value:string,electronictext:string>,filingdate:string,applicationtypecategory:string,partybag:struct<applicantbagorinventorbagorownerbag:array<struct<primaryexaminerorassistantexaminerorauthorizedofficer:array<struct<name:struct<personnameororganizationnameorentityname:array<struct<personfullname:string>>>>>,partyidentifierorcontact:array<struct<name:struct<personnameororganizationnameorentityname:array<struct<personstructuredname:struct<lastname:string>>>>,postaladdressbag:struct<postaladdress:array<struct<postalstructuredaddress:struct<addresslinetext:array<struct<value:string>>,cityname:string,geographicregionname:array<struct<value:string>>,countrycode:string,postalcode:string>>>>>>>>>,groupartunitnumber:struct<value:string,applicationconfirmationnumber:string,patentclassificationbag:struct<cpcclassificationbagoripcclassificationoreclaclassificationbag:array<struct<ipofficecode:string,mainnationalclassification:struct<nationalclass:string,nationalsubclass:string>>>>,businessentitystatuscategory:string,firstinventortofileindicator:string,inventiontitle:struct<content:array<string>>,applicationstatuscategory:string,applicationstatusdate:string,officialfilelocationcategory:string,officialfilelocationdate:string,patentgrantidentification:struct<patentnumber:string,grantdate:string>>,prosecutionhistorydatabag:struct<prosecutionhistorydata:array<struct<eventdate:string,eventcode:string,eventdescriptiontext:string>>>,assignmentdatabag:struct<assignmentdata:array<struct<reelnumber:string,framenumber:string,documentreceiveddate:string,recordeddate:string,maildate:string,pagetotalquantity:int,conveyancetext:string,assignorbag:struct<assignor:array<struct<executiondate:string,contactorpublicationcontact:array<struct<name:struct<personnameororganizationnameorentityname:array<struct<value:string>>>>>>>>,assigneebag:struct<assignee:array<struct<contactorpublicationcontact:array<struct<name:struct<personnameororganizationnameorentityname:array<struct<value:string>>>,postaladdressbag:struct<postaladdress:array<struct<postaladdresstext:array<struct<sequencenumber:string,value:string>>>>>>>>>>,correspondenceaddress:struct<partyidentifierorcontact:array<struct<name:struct<personnameororganizationnameorentityname:array<struct<value:string>>>,value:string>>>>>>>>,sequencenumber:string>>,assignmenttotalquantity:int>,st96version:string,ipoversion:string>> COMMENT 'from deserializer')
ROW FORMAT SERDE
'org.openx.data.jsonserde.JsonSerDe'
WITH SERDEPROPERTIES (
'paths'='PatentBulkData')
STORED AS INPUTFORMAT
'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION
's3://test010010/test/'
当我在雅典娜(Athena)中运行查询时,它给我以下结果很好,没有任何问题:
使用此create table语句通过在S3位置指向JSON数据来创建表。如果这给出任何架构错误,则问题必须是实际的JSON文件,而不是您在问题中共享的文件。
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。