Import webpages behind paywall?

I would like to be able to save articles from a site where I am a paying subscriber. When EagleFiler imports the article, only the teaser content available to non-subscribers is fetched. Obviously EagleFiler is re-fetching from the web and not actually grabbing what is visible in the browser window. Is there a way to supply login credentials to EagleFiler? Or can it somehow read and/or use the cookie information from the browser so that the full article can be saved?

Thanks for a great product, I use it every day!

No, the login/session state is only in the browser; it cannot be transferred to EagleFiler. You could either use the browser to save the page as a Web archive or have it print the page and use ā€œSave PDF to EagleFiler.ā€

1 Like

Thank you for your quick reply.

Yes I had a feeling that was the case. It’s a shame, I save web pages as PDFs already because of EagleFiler’s excellent web2pdf. I like that it preserves links as clickable, something that the standard printed PDF does not. Also, using the print service will set the From field in EagleFiler to my user name and not the name of the site. Unfortunately I use firefox so there’s not an applescript solution ready at hand. Le sigh.

Well a printed PDF is still better than none I guess.

If you use Safari it will print PDFs with clickable links.

Hello, I’m new to EagleFiler and would like to be able to save certain articles from New York Times which I’m subscribed to that is behind a paywall. After almost 10 years since the answer above, have technology improved such that this is now possible with EagleFiler?

The Importing Private Web Pages help page explains the various ways to do this.

1 Like

Thanks, Michael. I like the ā€˜Print - Save as PDF to EF’ but unfortunately the website get truncated at the margin. I suspect it is WSJ (in this case) that is not playing nice.

I can’t edit my earlier post, sorry.

Safari - File-Save As - web archive seems to have a good result, which is what I am looking for, except there are a lot more mouse clicks.

There is a bookmarklet called ā€œArchive in EagleFiler (Web Archive)ā€ but it didn’t save the whole article, only what a non-subscriber can see, where as, Safari File-Save As web archive works better. Is the bookmarklet different?

Yes, it’s possible that something is wrong with their print stylesheet.

You can also create Web archives from Safari using EagleFiler’s system service. For example, use Select All and then press ⌘! (Command-Shift-1 on a US keyboard).

The bookmarklet is like the capture key in that EagleFiler will fetch the page from the server, so it won’t work for pages that don’t have permalinks or that require logging in.

2 Likes

Saving a paywalled web page, like from The Washington Post, via Safari’s File-Save As - Web Archive only seems to provide the same view as a non-subscriber would see. That’s entirely exclusive of any EagleFiler considerations; the Web Archive file when open by double clicking within Finder sends you to the non-subscriber view.

Export as PDF does give the proper view with all the content, albeit in a very large format. There doesn’t seem to be a way to set the size of the PDF within macOS. Not really a big deal.

The SingleFile extension seems to work, but apparently is limited to saving to the Downloads folder in Safari. Ideally, I’d like to save to the EagleFiler To Import folder. I guess this is a Safari extension restriction.

So, does anybody have some other suggestions?

1 Like

I should add one detail.

From EagleFiler’s Record Viewer, a Saved As Web Archive page shows the Sign In button like this:

If you click on that Sign In button, it does take you to the original web page, where you get the option to sign with your existing account info. A bit cumbersome, but it does work. Ideally, I’d like a less cumbersome approach. That may not be possible…

Does it work better if you turn off JavaScript before viewing the Web archive?

Did you try my suggestion above of using EagleFiler’s system service?

EagleFiler’s system service doesn’t seem to help in all cases. I’d also like to retain the graphics in these web pages.

But… Turning off JavaScript in Safari did seem to work in the few saved web archive files I tried. Great!

So, would resetting EnableJavaScript in EagleFiler to disable be what I should do? I’m guessing that setting only affects how a web archive or whatever is displayed, not how it is saved.

That should work when using the service in Safari (if you select the graphics, too). It should save the content of the page without the JavaScript.

Yes, the EnableJavaScript setting is only for display.

1 Like

Well, that’s odd.

Turning off JavaScript in Safari allows the web archive files to properly work either when opened from EagleFiler or directly through Finder by double clicking.

But! Disabling JavaScript through the EnableJavaScript setting does not work. I even quit EagleFiler and verified that the setting was correct in the EagleFiler prefs file before restarting EagleFiler. Opening the web archive from EagleFiler gives me the unsubscribed web view. That makes no sense to me.

No big deal - worst case is Exporting as PDF.

If you have JavaScript disabled in Safari when you first save the Web archive, that will make it save an archive with different content.

Disabling JavaScript after the fact, for viewing, often works, but it depends on how the page is designed. Sometimes there is junk over the page content that’s visible unless JavaScript is active.

I still recommend using the system service as a way to get just the content that you want, in Web archive format.

I was checking some older web archives I’d made with JavaScript turned on in Safari. When I viewed them again using Safari, I hit the paywall. When I turned JavaScript off in Safari and tried viewing the same files, no paywall.

Even with JavaScript disabled in EagleFiler, having EagleFiler open them in Safari - with Safari having JavaScript enabled - I hit the paywall.

But, I will go with your recommendations. Thanks!

Yes, the EagleFiler setting only affects display within EagleFiler.