finds.readers.readers

Miscellaneous reader tools

  • external requests

Copyright 2022, Terence Lim

MIT License

finds.readers.readers.requests_get(url: str, params: Dict | None = None, retry: int = 7, sleep: float = 2.0, timeout: float = 3.0, delay: float = 0.25, trap: bool = False, headers: Dict | None = {'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36OPR/38.0.2220.41'}, verbose: int = 0) → Response | None[source]

Wrapper over requests.get, with retry loops and delays

Parameters:

  • url – URL address to request

  • params – Payload of &key=value to append to url

  • headers – User-Agent, Connection and other headers dict

  • timeout – Number of seconds before timing out one request try

  • retry – Number of times to retry request

  • sleep – Number of seconds to wait between retries

  • trap – On timed-out: if True raise exception, else return False

  • delay – Number of seconds to wait initially

  • verbose – Whether to display verbose debugging messages

Returns:

requests.Response or None if timed-out or status_code != 200