There are five main features WebCopy (and Sitemap Creator) need based on user feedback and our own observations. In no particular order, these are making the product easier to use, supporting multiple downloads at once, being able to pause and resume a copy, JavaScript support and authentication. The current plan is to address three of the five in WebCopy 1.7, starting with authentication.
Since the earliest days of WebCopy, it has supported challenge authentication (where a web browser prompts you for credentials) and form based authentication (where you enter credentials into a web page). Almost all web sites use the latter approach and with WebCopy this can either be tricky to configure or impossible due to websites using interactive methods such as authenticators or captcha codes.
In order to resolve this, WebCopy 1.7 includes a new option named Log in use web browser, found in the Passwords property page.
When this new option is set, WebCopy will display a browser window when you copy a website to allow you to authenticate with the web site. Once authenticated, the cookies associated with the site are applied to WebCopy's crawler and copying will commence.
This new feature is currently available in 1.7 nightly builds, please let us know if this feature helps or hinders!
You can learn more about this feature and any caveats on the documentation page.
All content Copyright (c) by Cyotek Ltd or its respective writers. Permission to reproduce news and web log entries and other RSS feed content in unmodified form without notice is granted provided they are not used to endorse or promote any products or opinions (other than what was expressed by the author) and without taking them out of context. Written permission from the copyright owner must be obtained for everything else.
Original URL of this content is https://www.cyotek.com/blog/products/webcopy-1-7-web-browser-authentication?source=rss.