jriou/restockbot
Archived
1
0
Fork 0
Code Issues Pull requests Activity

Add store support for Micro Center (#2)

Browse source 
Local stores are set with an "storeid" param in the query string of the URL
and by a "storeSelected" cookie to avoid garbage in the query strings for
further requests. As product URL is a unique key in the database, Micro Center
is able to handle an URL with the store ID for every product. We can add this
storeid in the list of URLs to parse and job done. Every single Micro Center
local store are parsable.

Signed-off-by: Julien Riou <julien@riou.xyz>
This commit is contained in:
Julien Riou 2021-02-28 15:11:31 +01:00
parent 42f79c03d4
commit 2afd36584b
No known key found for this signature in database
GPG key ID: FF42D23B580C89F7
1 changed files with 6 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

7
parser.go
View file

@ -342,6 +342,11 @@ LET first_page = '` + url + `'
LET doc = DOCUMENT(first_page, {driver: "cdp"})
LET base_url = 'https://www.microcenter.com'
// determine store id or default to "shippable items" (online store)
LET detected_store_id = LAST(REGEX_MATCH(first_page, '&storeid=([0-9]*)', true))
LET store_id = detected_store_id ? detected_store_id : "029" // id of the online store if not found
COOKIE_SET(doc, {"name": "storeSelected", "value": store_id})
LET next_pages = (
FOR a IN ELEMENTS(doc, "div .pagination .pages a")
RETURN base_url + a.attributes.href
@ -361,7 +366,7 @@ LET results = (
LET available = LENGTH(ELEMENTS(el, "div .price_wrapper form .STBTN"))>0
RETURN {
name: name,
url: base_url + url,
url: base_url + url + "?storeid=" + store_id,
price: price,
price_currency: "USD",
available: available,