1. Enter the robot user agent in htaccess file
RewriteCond %{HTTP_USER_AGENT} (SemrushBot|AhrefsBot|BLEXBot|serpstatbot|MJ12bot|SeekportBot|crawler|spider|Amazonbot|bingbot|DotBot|Googlebot|Googlebot-Image|Googlebot-Mobile|Googlebot-News|Googlebot-Video|Mediapartners-Google|AdsBot-Google|AdsBot-Google-Mobile-Apps|bingbot|BingPreview|msnbot|Slurp|DuckDuckBot|YandexBot|YandexImages|YandexMobileBot|YandexVideo|YandexMedia|YandexBlogs|YandexFavicons|YandexWebmaster|YandexDirect|YandexMarket|YandexVertis|YandexAdNet|Baiduspider|Baiduspider-image|Baiduspider-mobile|Baiduspider-news|Baiduspider-video|rogerbot|Twitterbot|Pinterest|Facebot|LinkedInBot|AdsBot-Google-Mobile|MJ12bot|AhrefsBot|SemrushBot|DotBot|Exabot|360Spider|ia_archiver|Archive.org|Gigabot|TurnitinBot|ia_archiver|UptimeRobot|SiteBot|Linkdexbot|SeznamBot|BlexBot) [NC]
RewriteRule ^ - [F]
2. Create robots.txt file
User-agent: *
Disallow: /