python - How to write to a csv file on the local file system using PySpark -

-

i have code writing csv file on local filesystem error - ioerror: [errno 2] no such file or directory: 'file:///folder1/folder2/output.csv'

columns = [0,1,2,3,4,5,6,7,8,9] data1 = rdd1.map(lambda row: [row[i].encode("utf-8") in columns]) data1_tuple = data1.map(tuple)  open("file:///folder1/folder2/output.csv", "w") fw:     writer = csv.writer(fw, delimiter = ';')     (r1, r2) in izip(data1_tuple.tolocaliterator(), labelsandpredictions.tolocaliterator()):         writer.writerow(r1 + r2[1:2])

on local filesystem following directory exists - /folder1/folder2/. why throwing error , how can write csv file on local filesytem @ specific directory?

path argument open

either string or bytes object giving pathname (absolute or relative current working directory) of file opened or integer file descriptor of file wrapped

not uri. means code should follows:

with open("/folder1/folder2/output.csv", "w") fw:     writer = csv.writer(fw, delimiter = ';')     ...

Comments

Post a Comment

Popular posts from this blog

android - Why am I getting the message 'Youractivity.java is not an activity subclass or alias' -

-
Image
i created own android project , cut , pasted code exercise: http://androidexample.com/show_phone_contacts_in_autocomplete_suggestions_-_android_example%20/index.php?view=article_discription&aid=106&aaid=128# when build project no errors or warnings. when run project window error: annoyingly, can't make window smaller, can't click of buttons. cick 'x' @ top, close it. here's project structure: where getting 'autocompletemain' from? (note small c) class named autocompletemain. help. and have activity included in android manifest, in case you're wondering: <?xml version="1.0" encoding="utf-8"?> <manifest xmlns:android="http://schemas.android.com/apk/res/android" package="com.androidexample.autocompleteedittext" android:versioncode="1" android:versionname="1.0" > <uses-permission android:name="android.permission.read_phone_state...
Read more

java - Log4j2 configuration not found when running standalone application builded by shade plugin -

-
i have application when run maven log4j2 working: mvn exec:java -dexec.args=... but when run jar standalone application shows error: java -jar log: error statuslogger unrecognized format specifier [d] error statuslogger unrecognized conversion specifier [d] starting @ position 16 in conversion pattern. error statuslogger unrecognized format specifier [thread] error statuslogger unrecognized conversion specifier [thread] starting @ position 25 in conversion pattern. error statuslogger unrecognized format specifier [level] error statuslogger unrecognized conversion specifier [level] starting @ position 35 in conversion pattern. error statuslogger unrecognized format specifier [logger] error statuslogger unrecognized conversion specifier [logger] starting @ position 47 in conversion pattern. error statuslogger unrecognized format specifier [msg] error statuslogger unrecognized conversion specifier [msg] starting @ position 54 in conversion pattern. error statuslogger unreco...
Read more

python - How do I create a list index that loops through integers in another list -

-
i have list: sentences = ['the number of adults read @ least 1 novel within past 12 months fell 47%.', 'fiction reading rose 2002 2008.', 'the decline in fiction reading last year occurred among men.', 'women read more fiction.', '50% more.', 'though decreased 10% on last decade.', 'men more read nonfiction.', 'young adults more read fiction.', 'just 54% of americans cracked open book of kind last year.', 'but novels have suffered more nonfiction.'] and have list containing indexes of sequences of sentences in above list contain number. index_groupings = [[0, 1], [4, 5], [8]] i want extract specified sentence sequences in variable "sentences" using indexes in variable "index_groupings" following output: the number of adults read @ least 1 novel within past 12 months fell 47%.fiction reading rose 2002 2008. 50% more.though decreased 10% on last decade. just 54% of a...
Read more