Skip to content
This repository has been archived by the owner on May 12, 2022. It is now read-only.

Ошибка в разборе данных #4

Open
Delimer opened this issue Mar 3, 2017 · 1 comment
Open

Ошибка в разборе данных #4

Delimer opened this issue Mar 3, 2017 · 1 comment

Comments

@Delimer
Copy link

Delimer commented Mar 3, 2017

Пытаюсь выбрать все данные по визитам за пару дней
python metrica_logs_api.py -source visits -start_date 2017-03-01 -end_date 2017-03-03

Конфиг выглядит так
{ "token" : "...", "counter_id": "...", "visits_fields": [ "ym:s:visitID", "ym:s:counterID", "ym:s:watchIDs", "ym:s:date", "ym:s:dateTime", "ym:s:dateTimeUTC", "ym:s:isNewUser", "ym:s:startURL", "ym:s:endURL", "ym:s:pageViews", "ym:s:visitDuration", "ym:s:bounce", "ym:s:ipAddress", "ym:s:params", "ym:s:goalsID", "ym:s:goalsSerialNumber", "ym:s:goalsDateTime", "ym:s:goalsPrice", "ym:s:goalsOrder", "ym:s:goalsCurrency", "ym:s:clientID", "ym:s:lastTrafficSource", "ym:s:lastAdvEngine", "ym:s:lastReferalSource", "ym:s:lastSearchEngineRoot", "ym:s:lastSearchEngine", "ym:s:lastSocialNetwork", "ym:s:lastSocialNetworkProfile", "ym:s:referer", "ym:s:lastDirectClickOrder", "ym:s:lastDirectBannerGroup", "ym:s:lastDirectClickBanner", "ym:s:lastDirectPhraseOrCond", "ym:s:lastDirectPlatformType", "ym:s:lastDirectPlatform", "ym:s:lastDirectConditionType", "ym:s:lastCurrencyID", "ym:s:from", "ym:s:UTMCampaign", "ym:s:UTMContent", "ym:s:UTMMedium", "ym:s:UTMSource", "ym:s:UTMTerm", "ym:s:openstatAd", "ym:s:openstatCampaign", "ym:s:openstatService", "ym:s:openstatSource", "ym:s:hasGCLID", "ym:s:regionCountry", "ym:s:regionCity", "ym:s:browserLanguage", "ym:s:browserCountry", "ym:s:clientTimeZone", "ym:s:deviceCategory", "ym:s:mobilePhone", "ym:s:mobilePhoneModel", "ym:s:operatingSystemRoot", "ym:s:operatingSystem", "ym:s:browser", "ym:s:browserMajorVersion", "ym:s:browserMinorVersion", "ym:s:browserEngine", "ym:s:browserEngineVersion1", "ym:s:browserEngineVersion2", "ym:s:browserEngineVersion3", "ym:s:browserEngineVersion4", "ym:s:cookieEnabled", "ym:s:javascriptEnabled", "ym:s:flashMajor", "ym:s:flashMinor", "ym:s:screenFormat", "ym:s:screenColors", "ym:s:screenOrientation", "ym:s:screenWidth", "ym:s:screenHeight", "ym:s:physicalScreenWidth", "ym:s:physicalScreenHeight", "ym:s:windowClientWidth", "ym:s:windowClientHeight", "ym:s:purchaseID", "ym:s:purchaseDateTime", "ym:s:purchaseAffiliation", "ym:s:purchaseRevenue", "ym:s:purchaseTax", "ym:s:purchaseShipping", "ym:s:purchaseCoupon", "ym:s:purchaseCurrency", "ym:s:purchaseProductQuantity", "ym:s:productsPurchaseID", "ym:s:productsID", "ym:s:productsName", "ym:s:productsBrand", "ym:s:productsCategory", "ym:s:productsCategory1", "ym:s:productsCategory2", "ym:s:productsCategory3", "ym:s:productsCategory4", "ym:s:productsCategory5", "ym:s:productsVariant", "ym:s:productsPosition", "ym:s:productsPrice", "ym:s:productsCurrency", "ym:s:productsCoupon", "ym:s:productsQuantity", "ym:s:impressionsURL", "ym:s:impressionsDateTime", "ym:s:impressionsProductID", "ym:s:impressionsProductName", "ym:s:impressionsProductBrand", "ym:s:impressionsProductCategory", "ym:s:impressionsProductCategory1", "ym:s:impressionsProductCategory2", "ym:s:impressionsProductCategory3", "ym:s:impressionsProductCategory4", "ym:s:impressionsProductCategory5", "ym:s:impressionsProductVariant", "ym:s:impressionsProductPrice", "ym:s:impressionsProductCurrency", "ym:s:impressionsProductCoupon", "ym:s:lastDirectClickOrderName", "ym:s:lastClickBannerGroupName", "ym:s:lastDirectClickBannerName", "ym:s:networkType" ], "hits_fields": [ "ym:pv:counterID", "ym:pv:dateTime", "ym:pv:date", "ym:pv:firstPartyCookie" ], "log_level": "INFO", "retries": 1, "retries_delay": 60, "clickhouse": { "host": "http://localhost:8123", "user": "...", "password": "...", "visits_table": "visits_all", "hits_table": "hits_all", "database": "..." } }

Полученные данные и ошибки
`2017-03-03 14:42:03 MainProcess INFO ### DATA SAMPLE
2017-03-03 14:42:03 MainProcess INFO ym:s:bounce ym:s:browser ym:s:browserCountry ym:s:browserEngine ym:s:browserEngineVersion1 ym:s:browserEngineVersion2 ym:s:browserEngineVersion3 ym:s:browserEngineVersion4 ym:s:browserLanguage ym:s:browserMajorVersion ym:s:browserMinorVersion ym:s:clientID ym:s:clientTimeZone ym:s:cookieEnabled ym:s:counterID ym:s:date ym:s:dateTime ym:s:dateTimeUTC ym:s:deviceCategory ym:s:endURL ym:s:flashMajor ym:s:flashMinor ym:s:from ym:s:goalsCurrency ym:s:goalsDateTime ym:s:goalsID ym:s:goalsOrder ym:s:goalsPrice ym:s:goalsSerialNumber ym:s:hasGCLID ym:s:impressionsDateTime ym:s:impressionsProductBrand ym:s:impressionsProductCategory ym:s:impressionsProductCategory1 ym:s:impressionsProductCategory2 ym:s:impressionsProductCategory3 ym:s:impressionsProductCategory4 ym:s:impressionsProductCategory5 ym:s:impressionsProductCoupon ym:s:impressionsProductCurrency ym:s:impressionsProductID ym:s:impressionsProductName ym:s:impressionsProductPrice ym:s:impressionsProductVariant ym:s:impressionsURL ym:s:ipAddress ym:s:isNewUser ym:s:javascriptEnabled ym:s:lastAdvEngine ym:s:lastClickBannerGroupName ym:s:lastCurrencyID ym:s:lastDirectBannerGroup ym:s:lastDirectClickBanner ym:s:lastDirectClickBannerName ym:s:lastDirectClickOrder ym:s:lastDirectClickOrderName ym:s:lastDirectConditionType ym:s:lastDirectPhraseOrCond ym:s:lastDirectPlatform ym:s:lastDirectPlatformType ym:s:lastReferalSource ym:s:lastSearchEngine ym:s:lastSearchEngineRoot ym:s:lastSocialNetwork ym:s:lastSocialNetworkProfile ym:s:lastTrafficSource ym:s:mobilePhone ym:s:mobilePhoneModel ym:s:networkType ym:s:openstatAd ym:s:openstatCampaign ym:s:openstatService ym:s:openstatSource ym:s:operatingSystem ym:s:operatingSystemRoot ym:s:pageViews ym:s:params ym:s:physicalScreenHeight ym:s:physicalScreenWidth ym:s:productsBrand ym:s:productsCategory ym:s:productsCategory1 ym:s:productsCategory2 ym:s:productsCategory3 ym:s:productsCategory4 ym:s:productsCategory5 ym:s:productsCoupon ym:s:productsCurrency ym:s:productsID ym:s:productsName ym:s:productsPosition ym:s:productsPrice ym:s:productsPurchaseID ym:s:productsQuantity ym:s:productsVariant ym:s:purchaseAffiliation ym:s:purchaseCoupon ym:s:purchaseCurrency ym:s:purchaseDateTime ym:s:purchaseID ym:s:purchaseProductQuantity ym:s:purchaseRevenue ym:s:purchaseShipping ym:s:purchaseTax ym:s:referer ym:s:regionCity ym:s:regionCountry ym:s:screenColors ym:s:screenFormat ym:s:screenHeight ym:s:screenOrientation ym:s:screenWidth ym:s:startURL ym:s:UTMCampaign ym:s:UTMContent ym:s:UTMMedium ym:s:UTMSource ym:s:UTMTermym:s:visitDuration ym:s:visitID ym:s:watchIDs ym:s:windowClientHeight ym:s:windowClientWidth
0 chrome 0 WebKit 537 36 0 0 30066 56 0 1488484141656852122 180 1 421224 2017-03-02 2017-03-02 22:48:55 2017-03-02 22:48:55 1 https://... 0 0 [0] ['2017-03-02 22:49:58'] [14196793]...

....

2017-03-03 14:42:03 MainProcess CRITICAL Iteration #1 failed
Traceback (most recent call last):
File "metrica_logs_api.py", line 127, in
integrate_with_logs_api(config, user_request)
File "metrica_logs_api.py", line 107, in integrate_with_logs_api
raise e
ValueError: Code: 26, e.displayText() = DB::Exception: Cannot parse quoted string: expected opening quote: (at row 1)

Row 1:
Column 0, name: Bounce, type: UInt8, parsed text: "0"
Column 1, name: Browser, type: String, parsed text: "chrome"
Column 2, name: BrowserCountry, type: String, parsed text: "0"
Column 3, name: BrowserEngine, type: String, parsed text: "WebKit"
Column 4, name: BrowserEngineVersion1, type: UInt16, parsed text: "537"
Column 5, name: BrowserEngineVersion2, type: UInt16, parsed text: "36"
Column 6, name: BrowserEngineVersion3, type: UInt16, parsed text: "0"
Column 7, name: BrowserEngineVersion4, type: UInt16, parsed text: "0"
Column 8, name: BrowserLanguage, type: String, parsed text: "30066"
Column 9, name: BrowserMajorVersion, type: UInt16, parsed text: "56"
Column 10, name: BrowserMinorVersion, type: UInt16, parsed text: "0"
Column 11, name: ClientID, type: UInt64, parsed text: "1488484141656852122"
Column 12, name: ClientTimeZone, type: Int16, parsed text: "180"
Column 13, name: CookieEnabled, type: UInt8, parsed text: "1"
Column 14, name: CounterID, type: UInt32, parsed text: "421224"
Column 15, name: Date, type: Date, parsed text: "2017-03-02"
Column 16, name: DateTime, type: DateTime, parsed text: "2017-03-02 22:48:55"
Column 17, name: DateTimeUTC, type: DateTime, parsed text: "2017-03-02 22:48:55"
Column 18, name: DeviceCategory, type: String, parsed text: "1"
Column 19, name: EndURL, type: String, parsed text: "https://..."
Column 20, name: FlashMajor, type: UInt8, parsed text: "0"
Column 21, name: FlashMinor, type: UInt8, parsed text: "0"
Column 22, name: From, type: String, parsed text:
Column 23, name: GoalsCurrency, type: Array(String), parsed text: "["ERROR

, e.what() = DB::Exception
`

@Slach
Copy link

Slach commented Dec 9, 2017

Недавно формат выгрузки поправили, попробуйте еще раз

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants