The predecessor, WSL, "just worked" and it was more a less a linux experience for most practical purposes-- and certainly better than hoary old cygwin.
This caused a lot of people to believe they could just transition to WSL2, lead on by the promise of an even more performant linux experience. The documentation didn't say anything about complications from attempting this, so a lot of people just tried it as soon as they could, thinking it would go as smoothly as when they tried WSL. But nope... it many cases, it doesn't just work out of the box. There's network configuration and gateway issues, snags with vpn, and now this git repo corruption. When you look on git issues, it's just people randomly shot-gunning suggestions, some of which work, some of which don't. I think WSL2 was rushed out too early, or at least it's lacking a comprehensive troubleshooting guide to get it up and running.
File system corruption is unforgivable mistake, but FS work is really hard to get right. Even stuff like VMware corrupts shared folders ... wherever they try to bypass a driver translation layer. Just Google for “vmware shared folder corruption”
FS corruption is the worst as you lose total confidence in the product. This is a mega process escape that the WSL team would need to transparently detail why it happened, what the remedy is, and why it'll never, ever happen again.
My guess is it's the result of shutting down the virtual machine that runs the Linux kernel too soon, leaving writes to the virtual hard disk in an inconsistent state. This coincides with "shutdown /r /t 0" being able to cause it as well as blue screens or power loss. And explains why I've never seen it, despite using WSL2 on insider preview builds on multiple machines daily: I almost never shut down, only for updates and new builds.
That’s corruption. Data loss would indicate an entire fs transaction getting dropped entirely - but not breaking the principle of atomic ops. A write should either happen entirely or not at all.
Corruption is when you write data and you read back different data of the same size. Data loss is when you write data and you read back correct data of the smaller size or nothing.
Not sure what you mean by entire fs transaction. But Linux doesn't have transactional fs interface, so open, write, close can be interrupted at any point with result of just having a new empty file after open, being one of the valid outcomes after crash.
“Note that a successful write() may transfer fewer than count bytes”
It also says
“A successful return from write() does not make any guarantee that data has been committed to disk. On some filesystems, including NFS, it does not even guarantee that space has successfully been reserved for the data”
I also don’t think calling data loss due to writes that do not make it to disk “file system corruption” is correct. For file system corruption, the file system data structures have to be overwritten (e.g. the boot record or directory data structures)
Filesystems don't try to order writes to different files. So you get HEAD pointing to a truncated commit, or a commit pointing to a truncated blob.
The same happens if you have a power loss or kernel crash on Linux (as a kernel developer, it happened to me several times when testing freshly-committed code).
Interesting. I basically live at the VMware community forums and could not remember this issue...
FWIW I am a user moderator down there and VMware desktop products (Workstation/Fusion/Player) has my focus and as a result I basically read almost every post that would report this issue.
I just DDG'ed it and see one report from 2014 [0], ok some more from around 2008 when using google.
Looks like this was resolved in 2015 as that was the last time I see it being mentioned.
Reminds me of an infuriating old bug in VirtualBox where it wouldn't notice the size of a file had changed. Devs were working in some Windows editor then using Git within VirtualBox to commit changes, resulting in inexplicable trailing nulls and garbage turning up in the repo
Took quite some time to figure out what was causing it. "Everyone is somehow corrupting files except for me, wtf?" Magic filesystem translation layers always suck
Is this file system corruption? The article isn’t 100% clear it ‘only’ loses some data writes, but it also doesn’t explicitly say files not being written are affected, directory structures are corrupted, etc.
So, to me, it looks like WSL2 not completely flushing writes to the underlying file system. Bad, but not as bad as file system corruption (which could lead to losing all data on the disk)
[ 1.956835] JBD2: Invalid checksum recovering block 97441 in log
And that's corruption that leads to log replay failing, i.e. rejecting it because honoring the replay in the face of checksum errors could make things much worse. Subsequently mount fails:
That's good because the purpose of journal replay is to make the file system consistent following a crash/power fail. And if the file system is dirty, replay is called for, but can't happen due to a corrupt journal, so now an fsck is required. i.e. it is in an inconsistent (you could say partly broken) state and needs repair.
I haven't seen syslog/systemd journal for other cases to know if there's instances of ext4 log replay that succeeds, but with missing files. That's not file system corruption, even if it leads to an inconsistent state in a git repository (or even a database). But this is still concerning, because to get a situation where log replay is clean but files are missing suggests an entire transaction was just dropped. It never made it to stable media, and even the metadata was not partially written to the ext4 journal.
qemu-kvm has a (host) cache setting called "unsafe". Default is typically "none" or "write back". The unsafe mode can result in file system corruption if the host crashes or has a power failure. The guest's IO is faster with this mode, but the write ordering expected by the file system is not guaranteed if the host crashes. i.e. writes can hit stable media out of order. If the guest crashes, my experience has been that things are fine - subsequent log replay (in the guest) is successful, because the guest writes that made it to the host cache do make it to stable media by the same the guest reboots. The out of order writes don't matter... unless the host crashes, and then it's a big problem. The other qemu cache modes have rather different flush/fua policies that can still keep a guest file system consistent following a host crash. But they are slower.
So it makes me suspicious that for performance reasons, WSL2 might be using a possibly volatile host side caching policy. Merely for additional data point, it might be interesting to try to reproduce this problem using e.g. Btrfs for the guest file system. If write order is honored and flushed to stable media appropriate for default out of the box configuration of a VM, I'd expect Btrfs never complains, but might drop up to 30s of writes. But if there's out of order writes making it to stable media, Btrfs will also complain, I'd expect transid errors which are also a hallmark of drive firmware not consistently honoring flush/fua and then you get a badly timed crash. (And similar for ZFS for that matter - nothing is impervious to having its write order expectations blown up.)
Thanks. That definitely is file system corruption. And that is very scary, as (assuming you have backups, which you should) losing files you’re working on is not the biggest problem you can have. That will lose you a few days at most.
Silent corruption of parts of the disk that you rarely access but still want to keep is scarier (you might have rotating backups for months or years and still eventually lose data)
So long as the file system is fixed, it should be straightforward to fix the git repository. I'm no git expert but maybe 'git repair' can deal with it; and if not then 'rm -rf' and 'git clone'.
To avoid silent corruption requires full metadata and data checksumming, ala Btrfs or ZFS. In those cases, not only is corruption unambiguously detected, but it's not allowed to propagate.
Not my area but I seem to remember bitches that Linux lies about fsync. As in it'll swear up and down that it flushed everything to disk, but it's lying.
Also over the years it seems like everyone I've seen that habitually edits files remotely ends up with this sort of pain and butthurt.
My experience was locked files or lost shared folders (VM doesn't see them until you say the magic words, aka randomly stop and start services until it works). Both happened so (relatively) frequently that I just lost confidence in the feature until version 16 where read-only folders works fine. (I stopped using it in version 9)
I could swear I ran into that corruption (or a similar one?) almost a decade ago. They still haven't fixed it?! It was quite reproducible too, it just happened when I transferred a large file...
That’s a shame. I have a high-powered desktop and sometimes it’s nice to work from the patio by opening a few VMs on my laptop. I get the oomph of the big box with the mobility of the laptop.
To be fair, this feature always felt..rickety. But it was very nice.
I'd also be missing the ability to plug in USB devices, which is very central to the work I do. RDP has some device passover support, but it's not even close to being able to replace vmware's USB support.
You are talking about redirecting USB via workstation as well? eg. connecting USB devices remotely to your VM? (asking as normal USB pass-through doesn't work for a shared VM, see [0])
I hadn't even considered that. Connecting USB devices remotely works well with vSphere, but never tried that with Workstation. They are going to completely removing the hostd engine from VMware Workstation. That would indeed also include that part and it is most likely not easy to get that working without hostd.
We (postgres) did fix an ENOSYS (missing syscall) problem at some point so WSL could run Postgres. The surprising thing for me was how long it took for anyone to tell us it was broken/spewing warnings. That was forced when we changed a warning to a panic.
What a bizarre claim. It's irrelevant if you don't need Postgres, a minor inconvenience if you can easily adopt a workaround, and a show stopper if you were relying on accessing a local Postgres instance.
Have you developed in this environment? I have developed more than 20 sites in WSL1, all with PG as the DB. I have it running in the same machine in windows for developemnt(which you are running otherwise you wouldnt be in WSL).Instead of using "localhost" you use "127.0.0.1" in your configuration, that's it.
I have scripts that rely on connecting to PostgreSQL via a UNIX socket, I couldn't use these scripts on WSL. A workaround wouldn't be too hard, but ideally WSL should be 100% Linux-compatible in my opinion.
No, I haven't. Thanks for clarifying, that does make it sound like much less of a problem. Though it might still catch out some people, e.g. on a corporate machine where you're permitted to run WSL but not Postgres.
Yes, that's what I do, and it is completely transparent. I dont know why people are so dumbfounded because I wrote it was a minor inconvenience at worst.
The level of inconvenience purely depends on your stack and how its developed. Often things which don't bother me have huge effects on other members of my team, or on people working on other projects.
Do you mean kernel modules? You can compile programs just fine. There's a whole infrastructure around using visual studio code to run the UI natively and compile things inside the linux environment.
The difference on I/O is enough to make WSL unusable.
I wasted a day trying to figure out if I had a problem with anti-virus or something that was blocking me before realizing that WSL I/O is just... well, slow.
People kept telling me to upgrade to WSL2 to solve that but the version of windows I had didn't allow it. Might have been a blessing in disguise given the data corruption bugs.
It was introducing in windows 10 release 2004. A lot of users myself included are still stuck on the 1909 update because it does not show up in our list of automatic updates. This usually happens if Microsoft update determines that some of your hardware may cause BSODs with the new update.
Rushing half-finished products out, offering little/zero support is the new Microsoft.
The reason $MSFT loves open source is because they can get press hype over projects that are 75% complete (which is the main goal), and they don't even need to support it, document it, or make it actually work.
Yeah I think you nailed it there actually. That's exactly their modus operandi.
What's even worse is they have managed to abstract most of the support away in this cycle. You can't get enterprise support now because they gutted that entirely. You can't get them to do anything on github because they keep moving all the projects around and erasing them all or auto closing the tickets and no one on first line support knows anything now other than how to reset a Microsoft account password.
> Yeah I think you nailed it there actually. That's exactly their modus operandi.
They're following the "rules" from "The Cathedral and the Bazaar", remember. Specifically, the "Release early and often" bit, for the purposes of this conversation. Microsoft are considered "good open source citizens" because of the changes they've made to follow the written non-rules as well as the the unwritten rules.
If you're going to fault Microsoft for following the rules, fault EVERYONE ELSE that does it as well.
I want people to realize that they crap on Microsoft hard for things that they gladly accept from other developers or other companies. The double standards in the IT community are absolutely insane.
I think Microsoft gets more shit exactly because people expect more of them. They are the giants. they aren't a scrappy startup that needs to release or go bankrupt.
Microsoft are seen as the safe choice, so their stuff has to "just work".
That's obviously a bad place for innovation within Microsoft, but i don't really think anyone cares about the future of Microsoft.
People having high expectations of Microsoft is not Microsoft's problem. Microsoft are human beings just like the rest of you, and just like any large company, organizational inefficiency handicaps skilled developers a great deal.
Base your expectations on reality, and you'll have a much better time.
It’s worth noting that people do crap on google for doing this style of stuff constantly. It has basically nothing to do with the release early part, and everything to do with how things are deal with after that early feature light release. The entire point of “release early” is to be able to communicate with users about what direction the project should go. If you don’t keep iterating and working with user feedback (as at least google often doesn’t), then that’s why people complain.
Half-finished releases haven been Microsofts SOP since forever. And especially for payware, bugs are only fixed in the next release, so you need to buy the subscription or the new release.
I don't think calling it WSL2 was a mistake. It wasn't something that was good for the users, but it had a very clear benefit for the people working on WSL: calling it WSL2 allowed them to close WSL1 issues en masse as "fixed in WSL2" and never look at them again.
It probably depends a lot on your specific use patterns, but I expect for most people this was a change that basically fixed a bunch of bugs, introduced negligible new issues, and had an identical interface.
I tried to build two projects in WSL1, one using the Z3 theorem prover, and one using Chrome for scraping. Both ran into kernel issues. So for me it failed about 100% of the time on anything non-trivial.
Z3 had a timer to stop the solve if it takes to long and that used a specific option of clock_gettime that wasn't supported. I hacked around this and it otherwise worked fine.
True, but if MS hadn't presented WSL2 as a variant of WSL, any issues fixed in WSL2 wouldn't count as fixed in the WSL issue tracker. I would prefer it if their issue tracker marked these, more honestly in my opinion, as "This issue is will not be fixed in WSL, you can migrate to HVL instead" (using HVL as a hypothetical name for WSL2) with a separate HVL issue tracker.
Whether they are counted as fixed in the WSL tracker is completely up to them. "Fixed, use HVL" is just as valid way to close a WSL1 issue ticket as "Fixed, use WSL2".
A poor argument IMHO. A naming change for the sake of an issue tracker that appears to be a net negative for users is not a wise choice. Naming and branding doesn't exist to serve the project's management tools.
I guess I don't follow your point. The very reason WSL2 exists is because there were countless issues that COULDN'T be fixed with the way WSL1 was implemented. Why would they leave an issue open they fixed, just because the fix required a complete re-implementation? Furthermore why would they change the name, this is literally how they are carrying forward the functionality of WSL 1. It's still Linux on Windows, there is still a custom subsystem to allow the functionality. It is quite literally still Windows Subsystem for Linux. As documented:
They aren't carrying forward the functionality of WSL1. Yes, there are issues that cannot be fixed in WSL1. There are also issues that aren't, and quite likely can't be, fixed in WSL2, that do work in WSL1. The file system corruption that happens here in WSL2 is a nice example, it is something that could not possibly ever happen with WSL1 because of the way it was designed. WSL2 is not and will never be a full replacement for WSL1; WSL1 and WSL2 are two separate products, both with their own advantages and disadvantages, and I wish Microsoft would treat them as such.
> Should Mac Office not be called Office because they completely re-wrote it?
I do not know how different Office for Windows and Office for Mac are, but to go with a different example, yes, I do think Visual Studio for Mac and Visual Studio Code should not have carried the Visual Studio name, it causes unnecessary confusion.
I was using WSL to do esp8266 development so I could use linux tools. The official esp8266 windows toolchain is based on cygwin. If I'm using something that needs a unix environment anyway, why use cygwin when you have WSL?
I upgraded to WSL2 because well 2 is bigger than 1 so it must be better. But no, nothing worked. Serial ports are not supported in WSL2.
I agree. I tried wsl2, and while it's nice, it has issues wsl1 didn't have. For instance, networking almost never worked until I applied a common workaround of resetting the ip stack. Wsl1 always worked fine for that. It's just not ready for primetime yet.
Same difference in my experience - WSL1 git screwed up git repos for me, broke git lfs, +++. I guess it’s more of the same-ish on WSL2, just different edge cases due to different edges.
There are other issues, too. At least at some point, absolutely no haskell based apps would run, since their stdlib used some syscall which WSL1 did not implement. Broke pandoc for me. Stopped bothering with WSL1 there and then.
Other issues which colleagues encountered include abysmal performance and broken python installations as PATHs and other environment details are wildly mixed inside WSL.
I just don’t understand why some people were seemingly happy with WSL1, there were so many rough edges. WSL2 is much much better in my experience, on virtually all fronts.
It didn't have bad IO performance. It had the same IO performance as Windows.
The problem is software that is badly written and does bad assumptions, like that continuing opening/closing files is good just because in Linux is good, that maybe true on most UNIX systems but nobody said that.
I think that WSL2 is a very very bad idea, you are no longer making a POSIX subsystem of Windows, a way to use the POSIX API in the Windows kernel, without any emulation (basically the same thing as WINE), you are running a virtual machine.
I would say that WSL2 performance is very bad if you work in the Windows filesystem. Sure, if you work from the WSL home directory that is mounted in a ext4 virtual filesystem performance is good, it's a VM.
But this is useless, you see the main advantage of WSL over having a VM or a dual boot was integration with Windows, the ability to use bash scripts to manipulate your Windows files, the ability to launch Windows executables and pipe the output into a POSIX executable.
All of that is useful if there is a strong connections between the two systems, if I can work with WSL in the same home directory as Windows where I have all my files. How is useful if before working on something (that could be a stupid thing like running a script to rename a bunch of files) I have to first copy the files that I intend to work on in the WSL home, run what I have to turn, and copy them back? And what if I want my IDE in execution in Windows with the project in Windows and I want to launch on the project bash scripts?
I hope they will not discontinue WSL1! If they will discontinue WSL1, unfortunately I will have to go back to cygwin that was not great but worked mostly fine, since I need integration between Windows and Linux.
If you drop WSL then you get confused branding of what they are. WSL1 and WSL2 make it pretty clear you're getting the Hyper-V thing for the latter and the former is a Linux sys call API layer.
I'm actually surprised they can't be used together.
There's a few exclusive portions, like the executable load error handler that triggers ELF to load under the subsystem, and the binding of the 'bash' executable. But mostly, to prevent a great deal of confusion.
Agreed. Whatever the implementation the name indicates the next step in this solution. To do otherwise would be like naming windows95 something other than windows after windows3.1. Marketing.
Docker support was the main reason to upgrade in my case. IIRC, WSL was missing some key functionality that made Docker unusable for my purposes. Of course I don't remember the details; all I know is I had no real choice.
The last job I had where I had a Windows desktop (about a decade ago, now) I used Cygwin extensively and never had any big issues. That includes running X11 not just shell stuff. It was quite solid.
The predecessor, WSL, "just worked" and it was more a less a linux experience for most practical purposes-- and certainly better than hoary old cygwin.
This caused a lot of people to believe they could just transition to WSL2, lead on by the promise of an even more performant linux experience. The documentation didn't say anything about complications from attempting this, so a lot of people just tried it as soon as they could, thinking it would go as smoothly as when they tried WSL. But nope... it many cases, it doesn't just work out of the box. There's network configuration and gateway issues, snags with vpn, and now this git repo corruption. When you look on git issues, it's just people randomly shot-gunning suggestions, some of which work, some of which don't. I think WSL2 was rushed out too early, or at least it's lacking a comprehensive troubleshooting guide to get it up and running.