> so why use vim?

vim is lightweight and miles ahead of ed. It's easier for me, as a user, to use vim than to use ed. It's simpler and loads faster than Notepad++ or Code.

> from the same problem that threads in general arose - some part of the program blocking another part of the program, making for a bad experience.

While I concur that a web site not loading makes for a bad experience, I do diverge on the solution.

> javascript runs in a single thread for the whole page - so anything that is blocking in any way, due to processing data, dealing with loading data, rendering any sort of 3d for you, mixing audio, really anything that would give you any sort of interactive experience could block the whole page from being responsive.

Solution: don't use javascript.

> you might as well as from what problem threads arose - same answer. it's true that we had fairly primitive cooperative multi-threading in c using lngjmp and setjmp, but it's still a lot better to fire up another thread when doing tasks like the above, especially when dealing with a gui is also involved, such as with the web

Honestly I don't see where the problem is a user problem. It's a problem, for sure, but I believe it rests solely in the hands of the developer and web site's servers.

> I'm happy to provide concrete examples, if you want.

I would like that, if you're willing to also learn how I think some of the examples might be done differently.

(we're too many levels deep on hn, so replying to myself - this may end up going beyond what would be a good discussion on hn though)

> vim is lightweight and miles ahead of ed.

and modern browsers are miles ahead of where they were 10 years ago, giving us better text rendering, better image rendering, much better layout, and better interactivity.

> Solution: don't use javascript.

for some things, this is acceptable, but for some applications this simply does not make sense. loading a static page, probably shouldn't be any javascript.

viewing a map, where you can zoom in and out, as well as pan around should probably use javascript. I'd be interested in hearing how you would solve that use case without using javascript.

> Honestly I don't see where the problem is a user problem. It's a problem, for sure, but I believe it rests solely in the hands of the developer and web site's servers. > > I'm happy to provide concrete examples, if you want. > I would like that, if you're willing to also learn how I think some of the examples might be done differently.

here's an example of something that I did real-world with web workers:

a city's building footprints, loaded from a feature service (~150mb as JSON), processed by a user-specific variable (such as age of building, square feet, building usage: store, warehouse, residential) where each building is rendered on an interactive map in different colors based on the visualization type. this can be done server-side, with 150mb of data rendered onto either vector or raster images, each processing tiles for any location, or it can be rendered in browser. without using web workers, expect a 5 minute freeze while everything loads, gets processed, and rendered into the dom, with the experience repeating every time criteria is changed or the map is moved. with web workers, it is near real-time, with no freeze. if this were rendered on the server, it would be quite a lot of server cpu usage to render on the fly, or a huge amount of image space to render ahead of time, plus very large amounts of data. add another layer on top of it, and you're looking at doubling that, along with huge data transfer, which would not make for a very good user experience.

I'd be interested in hearing an alternate way to provide this, while keeping the data usage, server disk usage, and cpu usage (server and client) to a minimum, while still providing a compelling user experience to the user.

> (we're too many levels deep on hn, so replying to myself - this may end up going beyond what would be a good discussion on hn though)

Where would you like to move the conversation?

> viewing a map, where you can zoom in and out, as well as pan around should probably use javascript. I'd be interested in hearing how you would solve that use case without using javascript.

Using a browser-only solution? First, I'd ensure that navigation buttons (forms) are available which could load a new page if the browser doesn't support features. Then, I'd augment HTML.

First, declare an image tag. <img>. Then add an attribute indicating that it's a sub-view of a larger logical image which can be stitched together (panning), and another attribute indicating it can be zoomed in/out; the attributes should include urls to retrieve new/additional images. If the user pans or zooms too far, the browser can either deny the change that the user tried to do (eg, the image stops panning/zooming) and/or present information to the user indicating why it can't fulfill the request.

This is, in my opinion, a superior solution since it explains to the browser exactly what it is that you want to allow the user to do. Then we don't need to require the user to run (unaudited, unsigned, untrusted) javascript code.

> a city's building footprints, loaded from a feature service (~150mb as JSON), processed by a user-specific variable (such as age of building, square feet, building usage: store, warehouse, residential) where each building is rendered on an interactive map in different colors based on the visualization type.

That sounds cool other than (I assume) 150mb being downloaded onto the user's device before anything can be rendered. I'd build a native application or a server-side application. I also hope that 150mb of data isn't sensitive to whether or not a user inspects it (no secrets or privileged information, for example).

> it can be rendered in browser. without using web workers, expect a 5 minute freeze while everything loads, gets processed, and rendered into the dom, with the experience repeating every time criteria is changed or the map is moved.

First: I think it's audacious to assume that every machine has a lot of CPU resources such that adding web workers magically makes things faster. If a user's device has a single core, then adding web worker's isn't going to make anything magically faster unless the only reason it's slow in the first place is from synchronous communication and execution.

Second: exactly how big is this DOM? When I think of a DOM, I think of a page or two or maybe three at the most. If so much data is being packed into the DOM that it takes minutes for the browser to render it then the DOM is being used for too much. Render an image of it on the server, calculate hotspots that a user could click on, and send the image and hot-spot regions to the client using an HTML map tag. Server then updates the image and sends a new one to the client.

Third: become a VNC server, start a session, and render it in the client using the same technique described above. Speaking from experience, if VNC is less resource intensive and faster then something is going wrong in the development pipeline.

> if this were rendered on the server, it would be quite a lot of server cpu usage to render on the fly

That's a problem... why? You'd rather push the hardware cost onto your user, I take it? I'd rather keep users' and consumer's buy-in costs as low as possible.

> or a huge amount of image space to render ahead of time, plus very large amounts of data. add another layer on top of it, and you're looking at doubling that, along with huge data transfer, which would not make for a very good user experience.

Have you measured it?

Also isn't 150mb considered a very large amount of data? Not everyone has awesome internet connections.

> I'd be interested in hearing an alternate way to provide this, while keeping the data usage, server disk usage, and cpu usage (server and client) to a minimum, while still providing a compelling user experience to the user.

Let me try to understand your project: you want you have an interactive map (ala Google Maps) with an overlay whose color is dynamic based on user selection?

It sounds like it could be done by using a regular HTML form to load images newly-rendered by the server. An added bonus would be allowing the user to navigate forward and backward among their selections using the browser's native navigation features.

Exactly why is it so expensive to render on the server? Exactly how much larger are these map images with data overlaid compared to the original map image? I would think that, generally, when you replace high-color-range data (eg, satellite images) with flat colors (overlays), you'd typically reduce image size (flat colors being compressible); regular map images (plain background, roads and features in solid colors, etc) are pretty small. JPGs and PNGs aren't very expensive (image size) at all compared to that 150mb. I would be extremely surprised if the user was pulling more than 150mb of images for a typical session navigating around a map and making analysis colorization selections.

If your server can't perform with basic image manipulation (take image, overlay a color, return result) then your server software and/or hardware needs some better developer/hardware resources assigned.

Regardless, it sounds like the issue you're trying to get at is the dynamic overlay. Again, I'm not a web developer, but doesn't HTML and CSS already support overlaying images on top of each other? How expensive do you think it would be to serve the underlying map images and, separately, serve pre-rendered overlay images?

I'd imagine the overlay images would be a single color so they should be really small. Not only that, but you can pre-render each building's footprint in black-and-white. Then when serving the pre-rendered image, change the color index/table/palette so that the foreground (black? white? doesn't matter) is whatever color you want to serve. I've not done image manipulation for about 15 years but GIF sounds perfect for this. Then the overlay just needs to go to the exact same coordinate as the underlying map image.

And, hell, just because I like to hate on javascript: let's go back to augmenting HTML. In the form tag, add an attribute to indicate that submitting the form should merely refresh a particular image with a specific name attribute. Then, when submitting the form to change the stats being shown, refresh overlay images only.

going to cut this one fairly short, as I think we're hitting the end of the discussion:

> Using a browser-only solution? First, I'd ensure that navigation buttons (forms) are available which could load a new page if the browser doesn't support features. Then, I'd augment HTML.

which doesn't give the user a very compelling solution, it gives them a static map; which definitely has its uses, but feature-wise isn't something that people would really like to use: we're a long way from Mapquest, and it's not a paradigm that users are interested in at this point, which is why browsers have expanded functionality to include javascript, and css.

> Have you measured it?

yes: each tile is 25k at low resolution, 100k at retina, times 15 (for zoom levels), 20 tiles for your average web view of a map. that's 2mb per zoom level. adding a single layer of information to that would add 2mb per zoom level. take the 3 possible options that I mentioned and you're at 6mb. add the ability to pan for a small city, and you're looking at ~2000 tiles, or 200mb (or 600mb to account for each option). add another layer, such as population density, and you're doubling it per layer added. add the ability to change the opacity between layers, and you're adding up to 255 time more data. that's significant, as we've now reached 153gb without adding an additional layer to give me something to look at while keeping that functionality. now store that, for each person just in case they come back or render it real-time.

>Also isn't 150mb considered a very large amount of data? Not everyone has awesome internet connections.

I'm on a 1.5mbit connection with 700ms of latency on a sunny day, 150mb of data for that type of application is much preferred to rendered images being delivered to me.

> If your server can't perform with basic image manipulation (take image, overlay a color, return result) then your server software and/or hardware needs some better developer/hardware resources assigned.

at scale. we altered map tiles real-time no problem, but it still made sense to cache them instead of eating up the cpu time for each request (in that use-case, all users got the same modification, but if we were to render all changes to the map on the fly, instead of minor changes in the browser, it would not have made sense).

> Third: become a VNC server, start a session, and render it in the client using the same technique described above. Speaking from experience, if VNC is less resource intensive and faster then something is going wrong in the development pipeline.

and then it becomes inaccessible to the majority of users. same with requiring the user to download an application to view what should really be a web page (could you imagine having an app for every web page you want to visit? that would just be silly).

I'm sure we can dive down further, but I'm not sure it's worth it - there are several problems that having javascript and a browser solve, you're welcome to attempt them without leveraging either javascript or the browser, but I'll happily continue to use web pages that are delightful to use, and leverage as many technologies as possible. I remember bbs's, and text-based interactive telnet sessions, as well as curses, and I think I prefer the modern web.

(quickly editing to add a comment about using css and html and just "render it that way" - which is what is occurring, but the amount of data available [yes, it's all public data] to give a compelling data-driven experience to the user still needs to be dealt with - that's where web workers come in, putting those images layered on top of the tiles, across 500,000 buildings takes time to process whether you're using javascript or some as-yet-to-be-invented way to composite those images in the browser: it takes time and it is better to render them in the background in another thread than it is to lock the browser for 20-45 seconds while it renders - this is progressive enhancement)