For private/testing purposes: just use one of the many CORS-proxies out there (e.g. https://crossorigin.me/, http://cors-anywhere.herokuapp.com/, https://www.google.com/search?q=corsproxy) or set up one locally for yourself. :)
On Thu, Dec 8, 2016 at 11:14 AM, Mutschlechner,Andreas ma@otetto.org wrote:
Hi,
having some free time, I thought coding a visualization about the budget of the autonomouse province of Trento. I'd prefer to code it for Southtyrol, but alas, those data are neither online nor does the minister of finance give them out.
Till now SOP, same origin policy, I encountered only theoretically, but now, trying to fetch data directly from source, does not work. Loosing some hours trying to circumvent SOP restrictions, I failed, but learned new things:) Of course, one solution is to download the data files onto my server and serve it from there. Another solution is, that the open data portal of Trento (well, in this cases the sources are on another server) sets the CORS HTTP Header "Access-Control-Allow-Origin: *" A third solution would be, that the data is delivered in JSON format, so I can use JSONP to get it.. There are other, almost crazy methods (with iframes and other strange DOM manipulation), which should make it possible to evada SOP restrictions, but ...
If anyone is eager to experiment with different methods to handle SOP, here's a base code snippet to start
<!doctype html>
<html lang="it"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <title>Bilancio Provincia Autonoma di Trento</title> <link rel="stylesheet" type="text/css" href="jquery.jqplot.css"> <script src="jquery-3.1.1.min.js"></script> </head> <body> <div id="page"> </div> <noscript> <span class="errmsg">Questo sito funziona solo con javascript abilitato.</span> </noscript> <script> ODPTsrcs = [ { title: "Riepilogo per macro aree 2012", url: "http://www.giunta.provincia.tn.it/binary/pat_giunta_09/opendata_grafici_2012/Riepilogo_per_macroaree2012.1416217960.csv", year: 2013 }, { title: "Riepilogo per titoli e categorie economiche ...", url: "http://www.giunta.provincia.tn.it/binary/pat_giunta_09/opendata_grafici_2012/Riepilogo_per_titolo_cat_economica_entrate2012.1416217961.csv", year: 2012 }, ]; $(document).ready(function() { ODPTsrcs.map(function(e) { $.get(e.url, function(d) { console.log("get"); }); }); }); </script> </body> </html> _______________________________________________ http://lists.lugbz.org/cgi-bin/mailman/listinfo/lugbz-list
Hi all,
Since I promised myself I'd just ignore your rants, I'll top post so that I can deny I actually wrote this reply later on.
I am violating my promise to just let your emails flow in my inbox as if they were from some namibian prince, because I would like to explain you in all honesty why I (and I believe quite some people in this list) avoid interacting with you.
Please follow my reconstruction of the thread:
On Thu, Dec 8, 2016 at 11:14 AM, Mutschlechner,Andreas ma@otetto.org wrote:
Hi,
having some free time, I thought coding a visualization about the budget of the autonomouse province of Trento. I'd prefer to code it for Southtyrol, but alas, those data are neither online nor does the minister of finance give them out.
Till now SOP, same origin policy, I encountered only theoretically, but now, trying to fetch data directly from source, does not work. Loosing some hours trying to circumvent SOP restrictions, I failed, but learned new things:) Of course, one solution is to download the data files onto my server and serve it from there.
Someone who optimistically think you are actually wiling to interact with the rest of the world, and that is kind enough to help you, reads your message and thinks:
`Hey! This guy lost "some hours" on a problem I know how to solve (or workaround), he'll surely like my help`
And proceeds to reply
On 2016-12-08 14:21, Martin Raifer wrote:
For private/testing purposes: just use one of the many CORS-proxies out there (e.g. https://crossorigin.me/, http://cors-anywhere.herokuapp.com/, https://www.google.com/search?q=corsproxy) or set up one locally for yourself. :)
So *I* would like to thank Martin for taking the time to reply and share his knowledge. I appreciated it!
But no, you are not interested in sharing ideas, you cannot just say thank you, no... You reply like this:
On 2016-12-08 14:39, Mutschlechner,Andreas wrote:
Proxying is a solution too, I did not try, because I'd be an idiot spending more time evading SOP, if I can just download the csv data files from the open data portal and put it on my server, so ...
So you basically say that you don't care about this enough to try (but hey, you knew about it already because you>>rest_of_the_world) but despite this you just dismiss it as "it could work"... then
Last thing yesterday spending time was trying the method using postMessage between iframes, but none does work on most common browsers.
How come you spent time on a different workaround that in the end doesn't work, you KNEW the solution that was suggested already...
You know what someone that was trying to help you read in such an exchange? I'll tell you what I would read if I was involved:
`So basically my proposal isn't worth his time, he claims he knows everything but he wasted "some hours" on suboptimal approaches, but... HEY, everything to not admit I gave a solution to his problem... I'll never ever interact with him again.`
Then your closing sentences:
Thank you for the interesting links. I am curious, for what you needed CORS-proxies.
Are perceived just like:
`yeah for sure I'll tell him, so that he can make jokes about how stupid my use case is and how better any approach of him is`
You know, if you read this far I am pretty sure this mail pissed you off enough that you are *not* willing to follow up, and this is exactly how quite some people feel and they end up avoiding this mailing list. And I say that because I talk with people offline and I know many that just stopped posting/answering because of *your* attitude.
Please try to put yourself in other people shoes when you interact with them, especially when they are helping you, for crying out loud!
With best regards, (doesn't this sound sarcastic, after all this mail? Trust me, it isn't. I am really trying to help.) Daniele
--
My answer is some code, as comment the shell skript, how I solved in the my problem I learned quite some bit about SOP and CORS.
Oder anders gesagt: Du hast Dich entschieden die Dateien direkt vom Trientner Server auf Deinen zu laden, wo auch Deine Webseite laeuft, so dass die SOP gar nicht mehr greift. Was Du auch bereits in der ersten Mail vorgeschlagen hast:
Of course, one solution is to download the data files onto my server and serve it from there.
Gut, dass das Problem somit fuer Dich geloest ist.
Aber: en-passant hast Du es in dem einzigen Thread geschafft:
- Martin anzuschnauzen ("I did not try, because I'd be an idiot..."), - den Ruf der LUGBZ in Frage zu stellen ("that LUGBZ just sucks..." etc), - Daniele einen "wheeny" zu nennen.
Im Gedaechtnis bleibt dann das. Nicht Dein Shell-Script, sondern Deine Ausfaelle.
Menschen, die eine duennere Haut haben als Daniele oder ich posten dann gar nicht mehr. Und das ist einfach nicht OK.
Ich erinnere an Shauns Mail:
http://www.lugbz.org/pipermail/lugbz-list/2016-July/023592.html
(und ja, dreh' mir bitte keinen Strick daraus, dass der Link zur Netiquette zur Zeit nicht geht, ich weiss es und hab's auf meiner To-do, die Netiquette mit Thomas' Verbesserungs- vorschlaegen (DANKE Thomas!) auf der neuen Seite zu veroeffentlichen).
No idea about you or Daniele
Fuer mich war neu, dass es oeffentliche CORS-Proxies gibt (DANKE Martin!).
Ansonsten - glaube ich - kennt jeder, der sich in den letzten Jahren mit Frontend-Entwicklung auseinander gesetzt hat die Problematik.
Bye, Chris.
Wie gesagt, wenn gewünscht wird, dass die Lugbz Mailingliste wieder das wird, was sie mal war, Dienstags ' Hallo Pizza essn', 1xjahr SFSCon, 1xHackathon, no comment question,......admin und vorstand können mit der liste machen was sie wollen..
Hi,
hierauf moechte ich doch eingehen, denn das sind einfach zu viele Fehlinformationen.
Diese Mailing-List ist eine offene und oeffentliche Mailing-List, auf der sich jeder einschreiben kann, egal ob Mitglied oder - wie Du - Nichtmitglied.
Genau deshalb moechte ich das Zerrbild, das Du hier vom Verein zeichnest, richtig stellen.
Als LUGBZ (Verein) haben wir 2016 diese Events veranstaltet:
- Progetto FUSS. Come configurare in modo semplice un laboratorio informatico - FUSS nelle scuole - Il Radioascolto con il Computer e i sistemi digitali di trasmissione usati dai Radioamatori - FUSS: creare un'aula informatica a basso costo con Raspberry Pi - R per l'analisi di dati e la loro presentazione: Cosa è R. Perché usare R. Quando usare R. - Open ma cifrato - Navigare (più) sicuri - WORKSHOP: Firefox Greasemonkey e Stylish: due addon open source per nerd e programmatori - WORKSHOP - Il server FUSS: struttura, filosofia progettuale e casi d'uso - Free Your Desktop toDay
Wir haben auch 2 social events veranstaltet (Grillen und Sternwarte Gummer) und ja, wir gehen auch ab und zu eine Pizza essen.
Die SFSCon und den Hackathon, den Du uns andichtest, veranstalten nicht wir, sondern das IDM. Bei der SFSCon sind wir zugegebenerweise immer vertreten mit dem SFSAward.
wer hier nach netiquette schreibt, wo ward ihr, als jemand schrieb, ich ticke nicht richtig? oder glatt behauptet ein Link geht, der nicht gehen kann?
Eben.
Das "tickt nicht richtig" Zitat ist eine der Hoehepunkt der ganzen konfusen Postings, die hier letzten Sommer das Klima vergiftet haben. Das vollstaendige Zitat war uebrigens
[Ironie beginn] ob dieser Herr mA wohl noch richtig tickt [Ironie ende]
15 Tage spaeter haben wir dann die Notbremse gezogen mit Shauns Mail, die ich bereits zitierte. Das hat dann meiner Meinung nach in etwa auch bis heute gehalten.
Bezueglich Link. War nicht *ich* es, der Dich verteidigt hat und gezeigt hat, dass der Link wirklich nicht ging und auch erklaert hat warum?
Bye, Chris.
<script> ODPTsrcs = [ { title: "Riepilogo per macro aree 2012", url: "http://www.giunta.provincia.tn.it/binary/pat_giunta_09/opendata_grafici_2012/Riepilogo_per_macroaree2012.1416217960.csv", year: 2013 }, { title: "Riepilogo per titoli e categorie economiche ...", url: "http://www.giunta.provincia.tn.it/binary/pat_giunta_09/opendata_grafici_2012/Riepilogo_per_titolo_cat_economica_entrate2012.1416217961.csv", year: 2012 }, ]; $(document).ready(function() { ODPTsrcs.map(function(e) { var url = e.url.split("/").slice(-1).pop(); $.get(url,function(d) { var a = d.split("\n").map(function(l) { return l.split("\t").map(function(m) { return m; });}); console.log(a); }); }); }); </script>
Sorry, ich beisse jetzt einfach an.
Reden wir ueber Code.
Was bitte hat das mit SOP zu tun? Du laedst Die CSVs ja nicht vom Trientner Server, sondern von Deinem eigenen. Was moechtest Du uns mit dem Code sagen?
Bye, Chris.
Aehm... jaja... :)
Reden wir ueber Code.
Was bitte hat das mit SOP zu tun? Du laedst Die CSVs ja nicht vom Trientner Server,
Also echt Chris, hast du echt nicht verstanden, weshalb ich Probleme mit dem SOP hatte? Verstehst du meinen Code überhaupt? Probier mal statt var url = e.url.split("/").slice(-1).pop(); var url = e.url, dann spielt SOP eine Rolle, doch so habe ich angefange-
Ja, ich versteh Deinen Code. Du schneidst den Filename raus und laedst nur den, als relativen Pfad von Deinem Server. Dort liegt das File, weil Du es vorher per C-Shell Script heruntergeladen hast.
Damit umgehst Du SOP, in dem du die Anfrage eben an die same origin schickst.
sondern von Deinem eigenen. Was moechtest Du uns mit dem Code sagen?
Hmm, wie gesagt, wärst du schnell gewesen, dann hättest du lachen können, wie ich mit map über ein array iterier, um die elemente unverändert zurückzugeben, also code, der nichts tut :)
Ach, a bissl extra Code stoert nicht :)
Du koenntest noch mehr vereinfachen:
"a/b/cccc".split("/").slice(-1).pop();
'cccc'
"a/b/cccc".split("/").pop();
'cccc'
Was soll ich schon sagen wollen? Isn't it beautiful whith how small code of javascript you can do many things without worrying of buffer overflows, type conversion, i even do not use try: except: constructs nor if or assert checks and still the code will work mostly reliable.
interessant auch das asynchrone programmieren in javascript, nicht?
Ach so. Du meintest das ganz allgemein. Ja, da hast du Recht. JS ist schon cool!
Bye, Chris.
Ja, echt cool, und jetzt wo der Anfang gemacht ist, du scheinbar auch schlaflos und zu viel Zeit hast, kannst du auch eine Visualisierung des Landeshaushaltes von Trient programmieren, ml open data hackathon :?
Ha, das wirkt vielleicht nicht so, aber ich arbeite gerade... :)
Fuer Hackathons habe ich eher nicht die Zeit (habe einmal einen mitgemacht)...
Bye, Chris.
PS: Python kann ich leider nicht. Du muesstest die Frage aber wohl in einen neuen Thread posten, den auf den hier wird kaum jemand antworten wollen :/
Andreas Multschlechner is now banned from this mailing list.
I wish everyone a nice weekend.
Kind Regards,
-- shaun
On 2016-12-09 05:47, Mutschlechner,Andreas wrote:
On 2016-12-09 05:17, Mutschlechner,Andreas wrote: don't feed the troll, but now it is anyway time to stand up (and sleepless nights I am used from my past as sysadmin)
there're some hidden messages in my posts. First I want to test if and how fast react the pubblic amministration of PAT, since I suppose, some people from there are on this list as well. So, if in some days the admins of http://www.giunta.provincia.tn.it/binary/pat_giunta_09/ set the CORS Allow..* header, absolutely great, compared to Southtyrol where those data are not online, or better, they are online in PDF format. Did anyone of you ever tried to read the bilancio provincial di aa? Come cittadino italiano io ho un diritto di ricevere i dati dall'amministrazione pubblica in un formato elettronica elaborabile, ma qua in AA il presidente e ministro delle finanze, non li pubblica, anche se durante le battaglie d'elezioni ha promesso trasparenza... e solo per ricordare, sua promessa del referendum costava solo 2mill euro...400.000€ speso per blackberry! Blackberry, ma per che cosa?????ed ora dove sono..
IMAO in my arrogant opinion va cosi tanto storto nella IT della provincia, che serve un Ercule per pulire questa stalla di Aughia ... (hahahaha, in RL gente della lista mi ha detto, che scrivo un pessimo italiano, peccato, a chi non capisce ;p
Buon giorno venerdi, andiamo a lavorare, erm parlare con gente in RL per un progetto per 2017-18 :)
scusatemi, se come solitamente vado contro diverse regole della netiquetta, ma mi permetto di fare marketing (per ora solo in tedesco) con un sito web, unusual (<ironically> he does not tick right</i>) per un altro progetto 2017 http://www.otetto.org/STIA/
@Chris, ci incontriame fra qualche minuto 'Mein Beck', Cappucino e Brioche 8*]]]] La vita e' bella, L'Italia e' il piu bel paese del mondo, ha detto Renzi, quando si e' dimesso./ _______________________________________________ http://lists.lugbz.org/cgi-bin/mailman/listinfo/lugbz-list
-
Chris Mair -
daniele -
Martin Raifer -
Schutte, Shaun