Allow only one file of directory in robots.txt? -

-

i want allow 1 file of directory /minsc, disallow rest of directory.

now in robots.txt this:

user-agent: * crawl-delay: 10 # directories disallow: /minsc/

the file want allow /minsc/menu-leaf.png

i'm afraid damage, dont'know if must use:

a) 

user-agent: * crawl-delay: 10 # directories disallow: /minsc/ allow: /minsc/menu-leaf.png

or

b)

user-agent: * crawl-delay: 10 # directories disallow: /minsc/*    //added "*" ------------------------------- allow: /minsc/menu-leaf.png

?

thanks , sorry english.

according the robots.txt website:

to exclude files except one

this bit awkward, there no "allow" field. easy way put files disallowed separate directory, "stuff", , leave 1 file in level above directory:

user-agent: *

disallow: /~joe/stuff/

alternatively can explicitly disallow disallowed pages:

user-agent: *

disallow: /~joe/junk.html

disallow: /~joe/foo.html

disallow: /~joe/bar.html

according wikipedia, if going use allow directive, should go before disallow maximum compatability:

allow: /directory1/myfile.html disallow: /directory1/

furthermore, should put crawl-delay last, according yandex:

to maintain compatibility robots may deviate standard when processing robots.txt, crawl-delay directive needs added group starts user-agent record right after disallow , allow directives).

so, in end, robots.txt file should this:

user-agent: * allow: /minsc/menu-leaf.png disallow: /minsc/ crawl-delay: 10

Comments

Post a Comment

Popular posts from this blog

android - Why am I getting the message 'Youractivity.java is not an activity subclass or alias' -

-
Image
i created own android project , cut , pasted code exercise: http://androidexample.com/show_phone_contacts_in_autocomplete_suggestions_-_android_example%20/index.php?view=article_discription&aid=106&aaid=128# when build project no errors or warnings. when run project window error: annoyingly, can't make window smaller, can't click of buttons. cick 'x' @ top, close it. here's project structure: where getting 'autocompletemain' from? (note small c) class named autocompletemain. help. and have activity included in android manifest, in case you're wondering: <?xml version="1.0" encoding="utf-8"?> <manifest xmlns:android="http://schemas.android.com/apk/res/android" package="com.androidexample.autocompleteedittext" android:versioncode="1" android:versionname="1.0" > <uses-permission android:name="android.permission.read_phone_state...
Read more

python - How do I create a list index that loops through integers in another list -

-
i have list: sentences = ['the number of adults read @ least 1 novel within past 12 months fell 47%.', 'fiction reading rose 2002 2008.', 'the decline in fiction reading last year occurred among men.', 'women read more fiction.', '50% more.', 'though decreased 10% on last decade.', 'men more read nonfiction.', 'young adults more read fiction.', 'just 54% of americans cracked open book of kind last year.', 'but novels have suffered more nonfiction.'] and have list containing indexes of sequences of sentences in above list contain number. index_groupings = [[0, 1], [4, 5], [8]] i want extract specified sentence sequences in variable "sentences" using indexes in variable "index_groupings" following output: the number of adults read @ least 1 novel within past 12 months fell 47%.fiction reading rose 2002 2008. 50% more.though decreased 10% on last decade. just 54% of a...
Read more

c# - “System.Security.Cryptography.CryptographicException: Keyset does not exist” when reading private key from remote machine -

-
i trying decrypt data using certificate private key. works fine when certificate installed on local machine (i using self signed certificate testing , have private key certificate) when try access private key remote machine using same code, "keyset not exist" exception. i using console application testing, , have made sure id have read permissions on private key on remote server. here sample code using: var store = new x509store(@"\\server1\my", storelocation.localmachine); store.open(openflags.readonly); var result = store.certificates.find(x509findtype.findbysubjectname, "server1.test.com", false); var certificate = result[0]; store.close(); //this succeeds both local , remote server var rsapublic = (rsacryptoserviceprovider)certificate.publickey.key; //this succeeds local, fails remote server var rsaprivate = (rsacryptoserviceprovider)certificate.privatekey; here exception call stack unhandled exception: system.security.cryptography.crypto...
Read more