Access to ciaobran

[chrisidso]

Hi Andrew,
Just tried to clone ciaobran and saw an error message that said my login did not allow me access to ciaobran.

I did check to see that I had all the python modules I needed, and I did have them.

how to I get access to ciaobran?

Chris

[datatalking]

Chris so what you will need to do is copy this to your GitHub and pull it down from there.

Let me know if that makes sense

[chrisidso]

That makes sense but I have no idea how to do that. Also, github has changed their access (since the last time I accessed my projects) so it now requires ssh keys. I was experimenting with that yesterday and got nowhere. So at this point the best I can do is clone the repository down to my machine and then run it, and play with it.
How do I copy it to my own github account?

Haven't tried on windows yet. I will need to install all of the python stuff mentioned in the readme including jupyter notebook in order to do that.

[chrisidso]

Andrew,
OK. Figured out how to copy to my github. Then was able to pull it down to my linux machine.

[chrisidso]

Andrew,
Working on this now. Sorry for the delay. I had a busy weekend and a small crisis to deal with this week. Working on a test plan now, then testing on linux.

[chrisidso]

OK. Andrew. I have some findings now. I put it through three rounds of tests. And I was only looking at the UI, and not so much at the underlying files/directories.
Round 1: Tried options 0,3.4,5, and 6 without first creating the tables. As expected all of them generated errors. No database or no tables to work with. In all of these cases main.py broke, and popped me back to the command line. It should not have done this. Should be able to handle an error and display a suitable error message and keep main.py running. Also the error message should be friendly to the user, instead of being the cryptic python error message. Probably need two levels of menus - one with just options 1 and 2 to start and after that add the other options.
Round 2: Option 1 first, then 0,1,3,4,5, and 6. With the tables created all of the other options work, except #6. Executing option 1 after creating the tables works too. It seems happy recreating all of the tables. Option 6 though does not work. And I could not get it to work at all. You'd think that you should type in the name of the backup file, so it can create it, but I got an error regardless of whether I typed the existing name or a new one.
Found something else at this point too. So, option 1, then 0 and change the name of the database. Then option #6. This will cause main.py to quit and pop you back to the command line. If you run main.py again, you will see that the database name has reverted to the default name (mstables.mysqlite). It should keep the new name though, until you decide to delete everything and start over.
I noticed that in some of the error messages it actually displays a list of the database names (contains default name and new ones). So it looks like when you change the database name it adds the new name to a list, rather than replacing the old name with the new one.

Round 3: Option 1, then option 2. Then 0,3,4,5,6 All the options work. #4 though displays a 'Goodbye' message and main.py quits, which is different than what happened in round 1. As with round 1, #6 did not work at all.

Then I tried deleting everything and then Option #2. As expected this generates an error message because there are no tables.

I just found something else. Option 1, then 2, and 2 again. I asked for 10 records in both cases. Seems to work, but also seems to have put the scraper into a weird state such that:
I executed options 3,4,5 to delete everything. Now , when I run main.py and choose option #4, it works. It deletes the tables, without an error message, even though I already deleted the tables. I can execute option 4 as many times as I want to and it will keep deleting the tables. Option #3 fails though because there are no tables. ?????

[datatalking]

@chrisidso these are good notes thank you for the process thinking, its helpful. When you put in the notes '#4' or '#6' are you actually referring to those issue numbers? Currently when you reference #3 it references (issue #3 on my screen a link pops up
Fail to initialize database, Win10 #3

Instead i'm thinking you are referring to the general repo general menu here [?]

So there was an update by a user pr 05957ad are you speaking to the same issue or different?

So with round 1 there should be a database that was created 05957ad was a PR that should have fixed this (should have I think)

Rund 2

With 'Option 1, then 2, and 2 again' the scraper itself doesn't access morningstar and I am looking at the option of adding in the data feed from yahoo, tiingo and others via pandadata reader but that is a sort of seperate PR with @caiobran permission or approval.

Round 3

So when you do '#6' menu option (is there a better way we should note that?) we should get feedback from @caiobran on how the backup is supposed to work. I've not dug deeper into how that was setup or maybe it needs to be address in another issue?

[chrisidso]

Andrew,
Correct. I was referring to the numbers of the menu items and not the existing issues.

I don't know if update 05957ad addresses the same issue. It may well be though.

About failure of option #6. I would create a separate issue for that.

[chrisidso]

Andrew,
So I am still trying to get set up on windows. I have installed everything I need (I think) but I haven'
t figured out yet how to get Jupyter Notebook to run. I'll keep you posted. And when I get it working I will test it.
Chris

[chrisidso]

Andrew,

You are right. Issue #4 does address the round one issue I mentioned above.
Do you want me to create a separate issue for the Menu Option #6 failure?

[datatalking]

Hi Chris, I'm just seeing this now. Can you pull or access the data from my githubrepo for #8 (comment)

…

On Wed, Feb 2, 2022 at 10:35 AM chrisidso ***@***.***> wrote: That makes sense but I have no idea how to do that. Also, github has changed their access (since the last time I accessed my projects) so it now requires ssh keys. I was experimenting with that yesterday and got nowhere. So at this point the best I can do is clone the repository down to my machine and then run it, and play with it. How do I copy it to my own github account? Haven't tried on windows yet. I will need to install all of the python stuff mentioned in the readme including jupyter notebook in order to do that. — Reply to this email directly, view it on GitHub <#8 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AC2ECWIEUA2W67NIILDMSE3UZF2N3ANCNFSM5M4YDYDA> . You are receiving this because you commented.Message ID: ***@***.***>

[datatalking]

Chris, please do post a seperate issue, I'm waiting to hear back from @caiobran

[datatalking]

@chrisidso I found the link I was looking for, https://github.com/joe-wojniak/PythonForFinance is the data source that can be used locally since the scrape isnt working any more.

I'll likely hear back from @caiobran soon, but what I am thinking of is creating a 'parsing tool' unless its already built for either IEX, Stooq or PandasData reader.

https://www.mssqltips.com/sqlservertip/6963/download-stooq-historical-stock-price-volume-data-sql-server/

I am still researching these

[chrisidso]

Hi Andrew,
I just created an issue for the menu #6 error. I included the entire error message too.

[chrisidso]

Still having difficulty on windows too. Cannot get jupyter to run and was unable to run main.py too. My computer did not seem to know what py or python meant. Still working on it.

[chrisidso]

I have not tried to pull the data from your github repo yet. Did not see the link to it in your earlier post. Can you re-post the link?