To do all this, the first thing that I needed to do is to auto login to this site. Fortunately, this site is still pretty high level and low budget and I didn't have too much trouble in reverse engineering it (the term does sound very impressive, but its just reading code). Unfortunately, even easy networking confuses me, and that is why I am here now trying to explain how to auto login a site here to save anyone a day of mindless trying.
- To emulate a browser in a script, I used the Httpurlconnection class in java because java is my most fluent programming language. In this class, the request and response headers are not very important, and if you do not know what they are, you have no need for them to make an auto login script. in the Httpurlconnection class, you could get your cookies through one of its function called get cookies(). After running this command, you could store everything inside a String array in accommodate for the multiple cookies that are present.
- With the cookies stored in a String array, the next important thing you would want to do is not to restore the cookies when redirected to the login page. Because of the nature of some site confining users in only using their login for a session, they would expire the cookie once it is set. To refresh the cookie would only make the cookie immediately expired. (Till this point, it had already been 5 hours for me)
- After storing the cookies and not reloading it after being redirected to the login page, you are logged in as long as you insert the first cookie from your stored list. For some site, they would like to add post parameters to add security. If so, just copy it from the networking tool from chrome because there is no way around it.
- Subsequent to your successful login, you would want to directly go to the site desired and get the html content you had always been wanting.
- Wala, you got data from a website from a script and could now automate data grabbing from websites. From this, you could notify yourself when your account had any changes or even storing your data on a visual program like rainmeter to remind yourself of your data.
Although I am aware that I did not explain everything in the utterly most simplistic way, I hope this information would help anyone that is stuck in their way of making an auto login script. And if anyone wanted to check out my version of my auto login script, they are welcome to check it out by downloading the zip or visiting it on GitHub. My program is also adapted to be a server so that I could receive and send requests to my main server for added efficiency.