20476010-Crawlomatic Multisite Scraper Post Generator Plugin for WordPress.zip

What Can You Do With This Plugin?

Crawlomatic Multisite Scraper Post Generator Plugin for WordPress is a breaking edge website crawling and scraping, post generator autoblogging plugin that uses website crawling and scraping to turn your website into a autoblogging or even a money making machine!
Get content from almost any webpage! You no longer need API’s which requires registration and provides limited access, also you can retrieve data from non API providing websites. Schedule it for once and let it autopilot your posts 7/24 for you like a master!

How does it work?

This plugin will crawl the seed URL you give it (crawling means that it will search all links that the webpage contains) and will visit and extract content from each crawled URL. The crawling process is customizable: you can set the crawling depth, crawling rate, maximum crawled article count, crawl only links with specific class or ID and many more customizations.

You can scrape content from almost every web site that you open in your browser. Just be sure that the content is not generated dynamically by javascript, such as load more buttons or lazy loading content by scrolling down.

Also, you can automatically generate unlimited number of custom website crawling and scraping.

Some usage examples:

• Coronavirus related websites
• Brexit related websites
• War related websites
• Super Bowl related websites
• The possibilities are endless!

Other plugin features:

• v1.6.5 update: Visual content selector support added – video tutorial
• v1.6.0 update: Added the ability to make screenshots of crawled pages and use them in generated post’s content – video tutorial
• v1.5.2 update: Ability to shorten outgoing (post source) links (and monetize them), using Shorte.st link shortener service – example of shortened link
• v1.4.8 update: Added JavaScript execution support for crawled pages – requires PhantomJS installed on server – How to install PhantomJs? – video tutorial
• v1.4.4 update: Added the ability to set multiple proxies for crawling pages. The plugin will select one at random at each page access
• v1.4.0 update: Added the ability to paginate crawling (crawling for articles will continue on the next page of the seed page).
• v1.4.0 update: Added the ability to import product prices for crawled products (WooCommerce compatible) + dropshipping price automatic modification – video tutorial
• v1.4.0 update: Added the ability to increase imported product price by a fixed number or to multiply it with a predefined number (great value for dropshipping!)
• v1.2.8 update: Added paginated post importing support (into a single crawled post) Check: VIDEO.
• v1.2.4 update: Added the ability to set proxies for crawling pages
• v1.2.3 update: Added an option to crawl the page from Google cache when direct crawling fails (blocked)
• Google Translate support – select the language in which you want to post your articles
• Text Spinner support – automatically modify generated text, changing words with their synonyms – built-in, The Best Spinner, SpinRewriter, WordAI, TurkceSpin and others – great SEO value!
• customizable generated post status (published, draft, pending, private, trash)
• shortcode to list all posts generated by this plugin: [crawlomatic-list-posts type => ‘any’, order => ‘ASC’, ‘orderby’ => ‘date’, ‘posts’ => 50, ‘category’ => ’’, ‘ruleid’ => ’’]
• automatically generate post categories or tags from marketplace items
• manually add post categories or tags to items
• choose if you want to update post if it is already posted
• send custom cookies with the request to the crawled webpage (authentification)
• generate post or page or any custom post type
• embeds videos from YouTube, Vimeo, Flickr, IGN, Ustream.tv and DailyMotion using website crawling and scraping
• define publishing constrains: do not publish posts that do not have images, posts with short/long title/content
• automatically generate a featured image for the post
• enable/disable comments, pingbacks or trackbacks for the generated post
• customize post title and content (with the included wide variety of relevant post shortcodes)
• ‘Keyword Replacer Tool’ – It’s purpose is to define keywords that are substituted automatically with your affiliate links, anywhere they appear in the content of your site. For example, you can define a keyword ‘codecanyon’ and have it substituted by a link to http://www.codecanyon.net/?ref=user_name anywhere it appears in your site’s content.
• ‘Random Sentence Generator Tool’ (relevant sentences – as you define them)
• option to automatically delete generated posts after a period of time
• detailed plugin activity logging
• scheduled rule runs
• custom field support for generated posts
• custom taxonomies support for generated posts
• unlimited crawled variable importing (unlimited imported parts of the crawled pages)
• option to copy or not images locally
• option to add canonical meta tag to generated posts
• Maximum/minimum title length post limitation
• Maximum/minimum content length post limitation
• Add post only if predefined required keywords found in title/content
• Add post only if predefined banned keywords are not found in the title/content
• Save and restore plugin rule list from file

Testing this plugin

• You can test the plugin’s functionality using the ‘Test Site Generator’. Here you can try the plugin’s full functionality. Note that the generated testing blog will be deleted automatically after 24 hours.

Plugin Requirements

• PHP DOM -> how to install it (if you don’t have it, but probably you already have it): http://php.net/manual/en/dom.setup.php
• PHP 5.0 or higher
• dom, mbstring, iconv and json extensions (enabled by default)

Need support?

Please check our knowledge base, it may have the answer to your question or a solution for your issue. If not, just email me at [email protected] and I will respond as soon as I can.

Changelog:

Version 1.0 Release Date 2017-08-15

First version released!

Version 1.1 Release Date 2017-08-16

Fixed some small issues

Version 1.2 Release Date 2017-08-17

Added the ability to crawl page by div class or id

Version 1.2.1 Release Date 2017-08-18

Fixed incompatibility with some WordPress installs

Version 1.2.2 Release Date 2017-08-22

Added a shortcode to display post generated by this plugin

Version 1.2.3 Release Date 2017-08-30

Added an option to crawl the page from Google cache when direct crawling fails (blocked)

Version 1.2.4 Release Date 2017-08-31

Added the ability to set proxies for crawling pages

Version 1.2.5 Release Date 2017-09-04

Added the canonicalization for generated articles

Version 1.2.6 Release Date 2017-09-13

Made the plugin timezone aware

Version 1.2.7 Release Date 2017-09-14

Fixed post date for non gmt blogs

Version 1.2.8 Release Date 2017-09-23

Added paginated post importing support

Version 1.2.9 Release Date 2017-09-27

Bugfixes

Version 1.3.0 Release Date 2017-09-28

Fixed rule restore

Version 1.3.1 Release Date 2017-10-20

Fixed featured image generation

Version 1.3.2 Release Date 2017-10-22

Added crawling helper

Version 1.3.3 Release Date 2017-11-06

Fixed a memory issue

Version 1.3.4 Release Date 2017-11-07

Bugfixes

Version 1.3.5 Release Date 2017-12-14

Fixed class selector not working in all cases

Version 1.3.6 Release Date 2017-12-18

Added the ability to specify a custom user agent for each crawled webpage

Version 1.3.7 Release Date 2018-01-20

Added a new text spinner service: Spinrewriter

Version 1.3.8 Release Date 2018-01-22

Plugin can now continuously import content

Version 1.3.9 Release Date 2018-02-02

Fixed issue when multiple crawl classes where specified

Version 1.4.0 Release Date 2018-02-22

Major update: added the ability to crawl imported product prices (WooCommerce compatible)
Added the ability to crawl serial content (paged crawling - crawling for articles will continue on the next page)

Version 1.4.1 Release Date 2018-03-07

Bugfixes

Version 1.4.2 Release Date 2018-03-21

Fixed a duplicate posting issue

Version 1.4.3 Release Date 2018-03-22

Fixed a critical issue with multiple rule running

Version 1.4.4 Release Date 2018-04-04

Added the ability to define multiple proxies. The plugin will select one at random at each page access

Version 1.4.5 Release Date 2018-07-13

Updated built-in readability module

Version 1.4.6 Release Date 2018-07-16

Critical bugfixes

Version 1.4.7 Release Date 2018-07-19

Added the ability to not translate links

Version 1.4.8 Release Date 2018-09-05

Added JavaScript execution support for crawled pages - requires PhantomJS installed on server

Version 1.4.9 Release Date 2018-09-18

Bugfixes

Version 1.5.0 Release Date 2018-09-24

Added the ability to add custom post taxonomies from crawled content
Added the ability to add unlimited crawled variables to posts's content/ meta/ taxonomies

Version 1.5.1 Release Date 2018-10-16

Fixed issue when importing large pages

Version 1.5.2 Release Date 2018-10-24

Added the ability to shorten links using Shorte.st

Version 1.5.3 Release Date 2018-10-29

Fixed issue when importing paginated posts

Version 1.5.4 Release Date 2018-11-06

Added the ability to strip HTML elements by tag name (div,a,span,etc.)

Version 1.5.5 Release Date 2018-11-07

Added WooCommerce product category creation support

Version 1.5.6 Release Date 2018-12-16

Added nested importing support - import mixed content into a single post, from multiple plugins created by CodeRevolution

Version 1.5.7 Release Date 2018-12-16

Added the ability to define a list of URLs to skip from crawling and importing

Version 1.5.8 Release Date 2019-01-08

Added the ability to import royalty free images for created posts

Version 1.5.9 Release Date 2019-01-12

Added Gutenberg blocks support

Version 1.6.0 Release Date 2019-02-01

Added the ability to make screenshots of scraped pages

Version 1.6.1 Release Date 2019-02-06

Improved compatibility with some crawled pages

Version 1.6.2 Release Date 2019-04-19

Security update

Version 1.6.3 Release Date 2019-05-15

Fixed some recently found bugs with post pagination

Version 1.6.4 Release Date 2019-05-17

Added support for TurkceSpin content spinner

Version 1.6.5 Release Date 2019-05-27

Added a much demanded new feature: Visual Content Selector for assigning scraped page content
Added the ability to scrape pages from bottom to top
Added the ability to replace words in scraped content
Other minor bug fixes and functionality improvements

Version 1.6.6 Release Date 2019-07-26

Fixed timeout issue with some crawled pages
Many small issues fixed and features improved

Version 1.6.7 Release Date 2019-08-05

Fixed issue with Google Translate

Version 1.6.8 Release Date 2019-11-15

WordPress 5.3 compatibility update

Are you already a customer?

If you already bought this and you have tried it out, please contact me in the item’s comment section and give me feedback, so I can make it a better WordPress plugin!

WordPress 5.4 Tested!

Disclaimer
Through this plugin you are able to grab content from various websites that does not necessary belong to you or which are not under your control. If you grab copyrighted material without the author’s permission, the plugin’s developer does not assume any responsibility for your actions. Also, the plugin’s developer has no control over the nature, content and availability of those sites.

Do you like our work and want more of it?
Check out this MEGA plugin bundle.