It had been long since I added any post on my work status. So, here is one ..
Well the week after the exam seemed quite confusing and loaded since I couldn’t proceed with most of the scripts due to one or the other obstructing errors I got. The major issue is the requirement of the script which in most of the cases is not properly mentioned, as such I have compiled the list of issues that ate this week of mine because of which i didn’t submit any important patch yet.
This is the list of the queries I had ..
Task one : Copyright repackaging.
- I had updated the pywikibot/compat/query file so that it may support full functionality of copyright. (basically by adding new utility functions – CombineParams, ConvToList, ListToParam and ToUtf8.
- But there is a warning given in this script, i.e., – this makes me doubtful if I am proceeding right.
WARNING: THIS MODULE EXISTS SOLELY TO PROVIDE BACKWARDS-COMPATIBILITY.
Do not use in new scripts; use the source to find the appropriate function/method instead.
Suggested : It’s preferred not to expand the content which are meant to support backward-compatibilty with compat. As such, this move was not preferred.
- I have added the scripts by making it compatible with the core version.
- Then I have added pywikibot/scripts/copyright folder with the following files:
copyright.py — output — which seem is working well but is without Google API or Yahoo API I was referring to, ending with.
copyright_put.py In this I am stuck at line 184 (I am not sure which ‘output’ or ‘pending’ file it is referring to) – please suggest me how can i create one such ‘output’ file so that testing might be completed. (I found this point while debugging it and concluding that here the program stops because the condition is not fulfilled.
copyright_clean.py — I get output which seems to be working.
- Additional files generated during testing – link
BESIDES, as most of them seem to be working I shall push these files namely : core/script/copyright/4 files + core/copyright/exclusion_list.txt.
Query 1 : Do I need to add something else too?
Suggested : I was suggested which files need to be added.
Task two : Missing possibility to retrieve images from a page that were not included through templates
For this what I understood is, getting a new function named say ‘linkedPagesthroughcontentparsing‘ in pywikibot/page.py and should make use of regex search operations to execute it . Isn’t it ? I am not very familiar with regex that’s why I have stopped proceeding. The link is what I have done till now by trying to see the implementation in compat version. Please suggest if I am on the right track. ( A lot of changes need to be done)
Suggestion : Need to follow pep8 guidelines and it’s better to have it as an argument like “content” which the default would be false and when it’s true you parse the content instead of the links.
Output but If I manually create the file I get :
>>➜ pywikibot-core git:(warnfile)✗ python pwb.py scripts/warnfile.py -lang:’test’ family:’test’ interwiki-bot.log
>> Parsing warnfile…
>> Fixing… 0 pages
where interwiki-log.bot is the log file generated using -log parameter with python interwiki.py.
As you had told me to go through the interwiki.py script, I inferred that it would be better in case you may give me example of some existing warnfile files so that I may use it for testing purpose since this warnfile.py is acting as mere a module which is imported in interwiki.py (L2441 in interwiki.py) if it gets parameter -warnfile:filename . So any pre-existing example might be very helpful for me to proceed as then I may test the script and then submit the patch after properly testing it.
Suggested: I have been suggested to use this command to run the script.
python pwb.py interwiki -new -family:wiktionary -lang:en -dry -log -ns:14
- This script is supposed to split the log file (like interwiki-bot.log) but since I am not yet done with regex i guess that’s hindering me to proceed.
- Anyway what I inferred is that in the different log files i used (locally present in my repo) namely: interwiki.log, interwiki-bot.2.log, interwiki-bot.log, makecat-bot.log : it seems not been able to get any matching warning as it expects L29 . Please let me what warning is it and how I may generate this warning again and thus test this script properly before submitting.
Task five: Porting : piper.py (not listed in phabricator)
- Output — Initially it was working perfect but now I guess due to the new package added for copyright task it is spilling these messages . How may I get rid of these messages? Now it doesn’t work anymore
- Query 2: One more problem once I asked you earlier this issue: for piper.py as you said i moved the message to scripts/i18n/piper.py file and it’s working as expected just that i couldn’t understand the fact that for some scripts like blockpageschecker.py the i18n script has the same name but for different key in the message twtranslate seems to work but when i tried o change the key for message (dict) for piper it doesn’t work so finally the name of the msg script is piper.py and the keys for the dict is also piper Why is it happening so ?
Suggested: To checkout how category.py in i18n directory works..
- Output — Seems to be working well.
- Please check the script once if I shall add something then i’ll submit the patch.
Suggested: It would be good if you use pagegen argument handling (so it supports something -cat or -start or etc.) (use genFactory etc.)
Query 3: One more thing I see is that i get message in most of the scripts:
>>>WARNING: /home/innovator/pywikibot-core/pywikibot/page.py:4751: UserWarning: Site test:test instantiated using different code “yi”
>>>link._site = pywikibot.Site(lang, source.family.name)
Is this fine or did I make some mistake or so?
Remark: It’s not a problem of the script.
So, this is the query list, since I have got their solution it’s easier to proceed.
Much for now..