How to grab web data into libre office calc ? Or Addons to grab specific web data into libre calc

Hi

Preamble ...skip if in a hurry

Just after posting the request I read the following page
https://help.libreoffice.org/Calc/Inserting_External_Data_in_Table_WebQuery#Editing_the_external_data

But that page is talking of entire pages

Not just a few cells for web pages

Or probably I haven't understood what's on
https://help.libreoffice.org/Calc/Inserting_External_Data_in_Table_WebQuery#Editing_the_external_data

Could you highlight the area(s) you desire then copy then paste to
wherever you wish the data?
            that's my system;
       [but I may not be understanding to what you're referring since I
have no clue as what smf is :wink: ]

Hi

please take a look at the image at
http://finance.groups.yahoo.com/group/smf_addin/

you will see it has 100s of data

copy paste would take ages

Hi :slight_smile:
You are asking in the right place and hopefully someone might have a good
answer for you or be able to point to places that might have more
detailed knowledge of what you are looking for.  Most of us started with Windows and many of us still use it a lot.

I tried a search for "smb" and then for "web" on our Extensions / Add-ons / Plug-ins site

http://extensions.libreoffice.org/extension-center?getCategories=&getCompatibility=any&sort_on=positive_ratings&path=%2FLibreOffice-Extensions-and-Templates%2Fextension-center&portal_type=PSCProject&SearchableText=web
but didn't find much.  It might be better to search in OpenOffice's
Extensions as many of them probably still work well or have maintainers
that might be interested in updating it in order to make it work in
LibreOffice.  Also the OpenOffice forums might be a good place to post
questions but if you do and post here as well then it's polite to let
both places know when you have found an answer and what the answer was.

Also most versions of Gnu&Linux (=distros) have their own forums and
mailing lists so it might similarly be good to post questions there in a similar way.  Which distro do you use at the moment?  Ubuntu/Mint,
Mageia, Fedora , openSUSE or which?  DistroWatch is a good place to
look-up the official forum/mailing-list for your distro
http://distrowatch.com
for example to find the Ubuntu ones try here (it redirects)

http://distrowatch.com/ubuntu
for Fedora find some good links here
http://distrowatch.com/fedora

There is also a general-purpose forum at

http://www.linuxquestions/org
that covers many, maybe all different distros but often you get answers from people using completely different distros, possibly even from different families so you sometimes have to figure out how to apply it to yours.  Most programs are either the same in each;  LibreOffice/OpenOffice,
Firefox, Thunderbird and many others.  Sometimes they choose slightly
different choices such as Chrome, Seamonkey, Opera instead of Firefox, similarly with utilities such as your  "package manager" so it doesn't take much to work out how to apply their advice and people at LinuxQuestions might even help with that.

Also there are other LibreOffice places such as AskLibO

http://ask.libreoffice.org/en/questions/

and other places you can find at

http://www.libreoffice.org/get-help/
but i still prefer this mailing-list.

Good luck and regards from

Tom :slight_smile:

Sort of.
http://getquote-tedsoft.blogspot.com/
and
http://extensions.openoffice.org/en/project/GetQuote

It is much more reliable with OpenOffice Apache than LibreOffice.

jonathon

Hi :slight_smile:
I liked the questions you asked in the Smf mailing list.  It might be worth asking them what language it's written in or if it just uses Excel macros or internal commands somehow.  LibreOffice Calc has both of course.

Also it'd be good to know if they have already licensed it or if they could consider making it OpenSource using a GPL or similar "copyleft" (a type of copyright) license.  Creative Commons explains it quite well for creative or artistic works or for documentation
http://creativecommons.org/
but they might understand the full proper GPL more easily because it's made for software
http://www.gnu.org/licenses/gpl.html
There are variants such as the LGPL, MPL and others that have slightly different nuances but i think the GPL is the best and most famous.

As OpenSource there are all sorts of opportunities for translating it between programs and maybe into different languages too, maybe getting a bit of a community together through their own site and/or the Extensions sites.

Regards from
Tom :slight_smile:

________________________________
From: Tom Davies <tomdavies04@yahoo.co.uk>
To: Supiramani Supiramani <hisupiramani@hotmail.com>; "users@global.libreoffice.org" <users@global.libreoffice.org>
Sent: Friday, 19 April 2013, 21:58
Subject: Re: [libreoffice-users] How to grab web data into libre office calc ? Or Addons to grab specific web data into libre calc

Hi :slight_smile:
You are asking in the right place and hopefully someone might have a good

answer for you or be able to point to places that might have more
detailed knowledge of what you are looking for.  Most of us started with Windows and many of us still use it a lot.

I tried a search for "smb" and then for "web" on our Extensions / Add-ons / Plug-ins site

http://extensions.libreoffice.org/extension-center?getCategories=&getCompatibility=any&sort_on=positive_ratings&path=%2FLibreOffice-Extensions-and-Templates%2Fextension-center&portal_type=PSCProject&SearchableText=web
but didn't find much.  It might be better to search in OpenOffice's

Extensions as many of them probably still work well or have maintainers
that might be interested in updating it in order to make it work in
LibreOffice.  Also the OpenOffice forums might be a good place to post
questions but if you do and post here as well then it's polite to let
both places know when you have found an answer and what the answer was.

Also most versions of Gnu&Linux (=distros) have their own forums and

mailing lists so it might similarly be good to post questions there in a similar way.  Which distro do you use at the moment?  Ubuntu/Mint,
Mageia, Fedora , openSUSE or which?  DistroWatch is a good place to
look-up the official forum/mailing-list for your distro

http://distrowatch.com
for example to find the Ubuntu ones try here (it redirects)

http://distrowatch.com/ubuntu
for Fedora find some good links here
http://distrowatch.com/fedora

There is also a general-purpose forum at

http://www.linuxquestions/org
that covers many, maybe all different distros but often you get answers from people using completely different distros, possibly even from different families so you sometimes have to figure out how to apply it to yours.  Most programs are either the same in each;  LibreOffice/OpenOffice,

Firefox, Thunderbird and many others.  Sometimes they choose slightly
different choices such as Chrome, Seamonkey, Opera instead of Firefox, similarly with utilities such as your  "package manager" so it doesn't take much to work out how to apply their advice and people at LinuxQuestions might even help with that.

Also there are other LibreOffice places such as AskLibO

http://ask.libreoffice.org/en/questions/

and other places you can find at

http://www.libreoffice.org/get-help/
but i still prefer this mailing-list.

Good luck and regards from

Tom :slight_smile:

________________________________
From: Supiramani Supiramani <hisupiramani@hotmail.com>
To: "users@global.libreoffice.org" <users@global.libreoffice.org>
Sent: Friday, 19 April 2013, 20:12
Subject: RE: [libreoffice-users] How to grab web data into libre office calc ? Or Addons to grab specific web data into libre calc

please take a

look at the image at

http://finance.groups.yahoo.com/group/smf_addin/

you will see it has 100s of data

copy paste would take ages

--------
regards

Date: Fri, 19 Apr 2013 13:02:08 -0500
Subject: Re: [libreoffice-users] How to grab web data into libre office calc ? Or Addons to grab specific web data into libre calc
From: laginnis@gmail.com
To: Hisupiramani@hotmail.com
CC: users@global.libreoffice.org

Could you highlight the area(s) you desire then copy then paste to wherever you wish the data?

that's my system;

[but I may not be understanding to what you're referring since I have no clue as what smf is  ;-)  ]

Hi

Preamble ...skip if in a hurry

--------------------------------------------

*I've been on the m$ office treadmill for 15 + years

*I've always been scared of Linux and thought it was all command line

*Recently stumbled on Linux and libre office ...so I am  completely new to

libre office

*Used it for a few days ... Truly it's a great piece of software and

dispelled most of my doubts on Linux, using Linux, libre office etc etc

Question / request reading

------------------------------------------

*However some specific doubts remain on libre office

*I've been using M$ XL's add on for grabbing web data , called

smf add in

*When activated, once could easily grab specific pages, specific cells of

web pages directly into xl ( without opening a web browser)

*This is a great time saver

*The only and only /  excellent  group for using smf add in  can be found at

http://finance.groups.yahoo.com/group/smf_addin/

*Some  basic write up on using smf add in can be found here

http://seekingalpha.com/instablog/326478-derek-a-barrett/1414481-automating-alpha-getting-started-with-stock-market-functions-smf-add-in-for-excel

Now, my question is , is there a similar add in for libre office, when

invoked , could help the user  get

specific cells ( not the whole web page )

Dear Tom

Thanks for the very very detailed and useful answers Tom. I thought I wrote detailed questions. But you went much further . Even better you wanted to see how SMF could get to open source .... You are just super

I shall ask in smf add in yahoo group. I am NOT sure if I will get any further because their starting web page stays no open office support. I am sure others might have asked over the years ...still I think they will tolerate one more question

I plan to ask the following ,.... But I'd appreciate if your could go thru these suggested questions and correct / add to them whee needed

1. What language is smf add in written in - my answer - must be XL VBA . the add ins are generally VBA, still ill ask
2. Is it open for porting to libre office / open office by others
3. ...anything else that I missed ....

Randy ( developer of SMF ) is generally a very friendly and very helpful person
I haven't seen anyone share and answer this much
I am not sure if he will allow open licensing ,.... Still I will ask

Wow !!

Thanks a ton for that Precise answer...

But having said that, this "..get yahoo quotes ..." is just a small portion of the smf add in

Let me wait and see if there are other answers

Hi

I'm planning to use Ubuntu

I forgot to add that answer on my earlier reply

Regards

Hi :slight_smile:
So the official forum is at
https://launchpad.net/ubuntu
along with a ton of other stuff.  Ubuntu is so huge that it easily supports several communities and there is another forum at
http://ubuntuforums.org/forum.php
and there is the cross-distro forum at
http://www.linuxquestions/org

The Windows way is to slash-and-burn, to completely lose the older system when you want to move to something new (or just different).  This means you get stuck with Win7 (or Win8 or whatever) before you have any idea how to use it.

The Gnu&Linux way is to install alongside so that you can keep using the older system so that you can keep meeting deadlines but allows you to test-drive the newer system and gradually become more familiar with it.

A good migration path into Ubuntu is to
1.  use some of the same apps in Windows.  So move away from Internet Explorer and start using Firefox (so you can still use IE if you need to), install LibreOffice/OpenOffice in Windows but keep whatever version of MS Office you use, Gimp instead of Photoshop and so on until you feel comfortable with change.

During that phase it's good to get an old machine out of a cupboard or something so you can try installing Ubuntu (or whatever) as a Dual-boot without losing Windows.  It's reasonably easy and has a fairly automatic path but it's still a good idea to have a practice.  Ubuntu is one of the few that has a cunning way of installing inside Windows but it's better to do a proper dual-boot.

2.  At first you are probably going to boot into Windows more than Ubuntu.  Don't worry about it.  Just push yourself into using Ubuntu to explore.

3.  Eventually you will start finding things are faster and often easier in Ubuntu although there are probably a couple of things you feel the need to tweak quite a bit.  There may still be one or 2 odd things that you haven't had time to figure out so it's easier to keep the Windows for those rare things.  Probably most of us have a Windows fairly near at hand, although outside of work i would be hard-pushed to find mine nowadays.  In work i probably use them about half-and-half.

When NASA find a new place far from Earth they don't start by trying a landing straight away.  They have a 3 step process with each step learning more and more until they know enough to have a reasonable chance of a smooth landing rather than just crashing
1.  Fly-by
2.  Orbit
3.  Test-landings

In most engineering disciplines it's generally considered a good idea to build a mock-up or prototype before launching into full-scale production.

So quite why the Windows world wants you to always abandon your working system and replace it with something completely new and different is a bit of a mystery.  It makes people nervous about trying the new thing if they can't return to their familiar system.

Regards from
Tom :slight_smile:

Hi :slight_smile:
The questions seem about right.

Your question 1 was mainly to find out if it is written as a macro or is just a clever worksheet or if it is more than a macro.  I get the feeling it's somethign clever in Java or maybe even C++ or something (C++ would be fantastic).

He doesn't seem to have protected his ownership with any sort of copyright yet.  He does seem to be a good chap that is just glad to see his code being used.  So, he might well be keen to protect his work by using the GPL so that some company can't swoop in and grab his work for their own profit and cut him out from being able to use any of that code elsewhere in other stuff he is working on.  There are other licenses he might prefer.  Creative Commons has a calculator for documentation or art or music but i don't think they have good ones for coding.

The questions are better the way you asked them because it sounds less threatening and gives him chance to give short answers.  We don't need to push him in any direction because Extensions are independent of the main code and his could have any license he likes. 
Regards from
Tom :slight_smile:

Thanks Tom

Detailed replies once again from you .... Much appreciated ....

I'm afraid I can't keep up with you !:slight_smile:

1. Posting at other forums ...what do you suggest ? Wait for a week and post same question elsewhere ? So that regular readers don't get piss3d off ? Or ? I will at least wait till end of weekend

2. Yes... I have tried this use parallel machines windows and Ubuntu side by side ... Ms office and libre side by side concept. I think this web page data grabbing is the main limitation as far as I am concerned ...my usage is concerned. The rest are small inconveniences . I don't k ow for sure. I may find more areas.

3. having two machines , dual boot etc etc ; i hope to hold on to the old windows machine for some months, try back and forth, decide if there are some very critical problems / areas not addressed by ubuntu etc. but the age of the old machine and other factors may finally decide it ( Dual system ) ... I may have to move to a new machine soon. On the new machine I will NOT buy windows and ms office because that seems costly to just experiment ..... the machines are just about the same. windows and ms office addres some $250 or so on top. so i do not plan to buy windows again on the new machine. so the age of the old machine will decide how long I can hold on to windows , ms office

4. My experiments so far tell me that ms office has these additional functions and a large VBA library and help forums which may be missing or which may be just under development in libre office etc ...that seems to be the catch

Thanks and regards

Hi :) 
1.  I was thinking more like a couple of days.  Sundays might be good to catch the weekend crowd and still catch the early week-days people.

2.  Do you need to grab all the time or can you leave the grabs to batch process them?  If rebooting into Windows is something you only need to do 1/day or something then it might not be so tough.  Eventually you will find some way around it.  Sorry no-one has dealt with this part of your question yet.

3.  Keeping the old Windows machine for a while makes sense but you might be able to just move the old hard-drive into your new machine.  On your new machine in Ubuntu get to a command-line to run
sudo update-grub
and then grub2 will find and add your old drive's version of Windows to it's boot-menu.  When you first boot into the old Windows it will freak out a bit and keep saying "Found new hardware".  It might take half an hour or so before it starts all that though and it's only a one-off for all the new hardware.

Typically new machines come with Windows included in the price and may even cost more without Windows.  It's a bit bizarre.  It's partly due to bulk purchasing and partly due to special deals made on the obligatory bundled software.  Some people call it "the Windows tax" because it means every new pc sale generates some income for MS regardless of whether it will ever get used or not.

Nowadays the version of MS Office that gets bundled in is typically trial-ware rather than full version.  So it has limited functionality or stops working after a short period of time.  It's a 'good' way of getting people hooked on using it and then 'forcing' them into paying the extra in order to get the full version.  In the unlikely event that you have the installer Cds for your old version of MS Office you might be able to install that on your newer machine although you'll need to phone MS to confirm that re-using your license is due to replacing your machine and getting rid of the old one.  They should allow it and sometimes they do.

The easier route is to just drastically shrink the Windows partitions and avoid booting into them at all.  Install Ubuntu (or whatever) as a dual-boot but just avoid using the Windows on that drive.  If you already had your old hard-drive from your old machine plugged in when installing Ubuntu then you old version of Windows will automatically appear in the boot-menu but if you add the drive later (my preferred way) then just updating grub adds it in.

4.  MS Office is horribly bloated with tons of stuff that 90% of users never use.  Apparently around 80% of MS Office features never get used by around 90% of MS Office users.  But they still need to hog resources (such as space in Ram) just in case you do need them.  The LibreOffice/OpenOffice way is to have plenty of add-ons to cover plenty of those extra things so you can add them if you want but not suffer from the bloat if you don't need all that unused stuff.  Then the Unix or Gnu&Linux way is to have independent 3rd party programs specialising in functionality such as Emailing or Calendar so that you can choose from amoungst competing packages to choose the one that most suits your needs rather than trying to fit into a one-program-suits-all.

As for VBA isn't that based on Basic which was popular in the 1980's?  LibreOffice/OpenOffice seem to offer a choice of 4 languages for macros.  I think Python is probably far better and more secure but apparently Javascript is another alternative.  I've not heard of BeanShell before.  If you really need Basic then LibreOffice/OpenOffice basic is different enough that the malware that macros are famous for don't work or are unlikely to work in LibreOffice.  I don't think MS Office lets you choose different languages for macros.  Python sounds like a much better option.

Regards from
Tom :slight_smile:

Hi :slight_smile:

Hi Tom and others !

1. I was thinking more like a couple of days.
Sundays might be good to catch the weekend crowd and
still catch the early week-days people.

sure

2. Do you need to grab all the time or can you leave
the grabs to batch process them?

batch process would do . grab data once in two days say .....

If rebooting into
Windows is something you only need to do 1/day or
something then it might not be so tough.

I plan to get out of windows totally ...but that is a plan

Typically new machines come with Windows included in
the price and may even cost more without Windows.

I didn't know this ... hmm... then my plan fails ! I can stop writing here :frowning: !

It's a bit bizarre. It's partly due to bulk
purchasing and partly due to special deals made on the
obligatory bundled software. Some people call it "the
Windows tax" because it means every new pc sale
generates some income for MS regardless of whether it
will ever get used or not.

I see

In
the unlikely event that you have the installer Cds for
your old version of MS Office you might be able to
install that on your newer machine

I do NOT have the old CDs

The easier route is to just drastically shrink the
Windows partitions and avoid booting into them at all.

ok

Install Ubuntu (or whatever) as a dual-boot but just
avoid using the Windows on that drive. If you already
had your old hard-drive from your old machine plugged
in when installing Ubuntu then you old version of
Windows will automatically appear in the boot-menu but
if you add the drive later (my preferred way) then
just updating grub adds it in.

I have to check IF there are web tutorials on how to do this ..... I know you have written a lot, but I'm lost when it comes to dual boot / dos commands / Cont + alt + T etc !!! :frowning:

4. MS Office is horribly bloated with tons of stuff
that 90% of users never use.

agreed

As for VBA isn't that based on Basic which was
popular in the 1980's? LibreOffice/OpenOffice seem to
offer a choice of 4 languages for macros. I think
Python is probably far better and more secure but
apparently Javascript is another alternative.

I din't know this. even IF I did , do I know how to use phython ? NO.. !! so ..., I've lost you on a lot of what is said after this line ! :frowning:

I've
not heard of BeanShell before. If you really need
Basic then LibreOffice/OpenOffice basic is different
enough that the malware that macros are famous for
don't work or are unlikely to work in LibreOffice. I
don't think MS Office lets you choose different
languages for macros. Python sounds like a much
better option.

Regards from

Tom :slight_smile:

Thanks and regards

Hi :slight_smile:
The "Websites Team"'s mailing list might know fancy ways of grabbing the appropriate data.

Have you been able to try out the Extension suggested earlier in the thread?  Here's a guide on how to install Extensions

https://wiki.documentfoundation.org/Documentation/HowTo/install_extension

Just out of curiosity, is there some word. phrase, set of numbers or some other identifier on lines you want to capture?  Some starting-marker?  Perhaps something to indicate end-of-line (or even end-of relevant bit)?

As for not being able to completely escape from Windows just don't worry as you are not alone!  Just because Windows is available on a machine doesn't mean you have to use it.

I'm not quite sure what you are hearing about dual-boot.  It's really not that complicated.  It's pretty much the default way of installing most versions of Gnu&Linux.  In Ubuntu  when you get to the "Partitioning" section you are given 3 choices
1.  Wipe Windows and everything else on the drive.  (obviously don't choose this one!  (Jic you ever need to return the machine to the shop or send it back to the manufacturers for some hardware defect.))

2.  Ubuntu and Windows (or whatever) side-by-side.  (this is the best option for noobs and is what i used first several times).  
3.  Something Else
I like the finesse i can get from the 3rd option but the 2nd one is fine.  I even know some experienced Gnu&Linux users that use it just for it's simplicity.  Just keep clicking on the "Next" button to let the installer work it all out for you.  The boot-menu and everything is sorted automatically.  It's really no more complicated than choosing English or some other language and is even less complicated than choosing your time-zone.

The stuff about VBA in MS Office vs the 4 options allowable in LibreOffice/OpenOffice is ignorable if you don't write macros.  MS keeps assuring us that their macros are safe now and can't be used as an attack-vector for malware but force people to accept a disclaimer that pops-up warning about the risk of running a macro and keep giving us "security updates" about it.  While the MS version of "Basic" has been traditionally used as an attacker vector the LibreOffice/OpenOffice variant of "Basic" has not ever been used to compromise systems afaik.  Plus you don't have to use "Basic" and could choose to use a variety of more popular and more secure proper programming languages.

Regards from

Tom :slight_smile:

Hi :slight_smile:
Did you ever get a satisfactory answer?  All my stuff seemed to be work-arounds or completely off-topic.  Hopefully helpful but not dealing with the original question. 
Regards from
Tom :slight_smile: