Death of a Clearcase May 07, 2011 at 02:37 AM | categories: clearcase, git | View Comments
Well it's been quite some time since my last post. Time has moved
on in many respects, and yet it also feels like very little has changed.
Someone at work suggested that I should add a final post to the saga of
migrating away from Clearcase at work.
The good news is: we won!
That is to say that we now use Git for a majority of projects at work, with the last few being migrated as we speak. Huzzah!
How did it happen?
Sadly my persistent nagging made very little difference in the end. Fortunately, however, money did; the cost of continued licensing of Clearcase was not attractive to the bean counters upstairs. Combined with the increasing globalisation of our team meant that Git, or at least a free DVCS, became a very real option (free being the operative word). Mercurial was the first choice, but by those who didn't have any hands-on experience with either; a mistake I have blogged about earlier. Given that my team and I had been using Git in-anger for at least 6 months we fortunately (or unfortunately) managed to convince them to think again.
We decided to host our source in The Cloud; specifically on GitHub. Now GitHub is an impressive site and I use it for all my open source projects. However, I do have one minor gripe: sadly GitHub has not yet implemented fine-grained security for Organisations. Given that the majority of users are open source and would most likely fork-on-write this makes sense. In our environment where we effectively work on a single repository it would be nicer if we could delegate a project leader or leaders and have them add/remove users for their team. Add the moment only 'Owners' can appear to do that, who can then also see billing information. The other problem is that with push authorisation you get force-push as well, which can be extremely dangerous in the hands of some. Not an ideal situation to be sure. Before the official migration we used the awesome gitolite which handled security beautifully, even down the branch/force push level. I would hope that at some point GitHub will adopt something more like that; but for now things are still 1 millions time better than using you-know-what and we can't complain too much.
I should add that the obvious rebuttal to this complaint is that we probably need to consider switching to a more distributed/pull-model of collaboration, where everyone has their own repository. I don't disagree with this necessarily, but having a push model gives us high visibility of changes and a quick development cycle. A discussion for another day perhaps.
And how about the users? Are they happy?
As much as I love Git, I'm not finding it any easier to teach some of the finer points of Git and DVCS, and people still ask me for help on a daily basis. It does take a while for the difference between checkout and reset to sink-in, which I can emphasise with. Also multiple branches from within one repositories is something that causes confusion to begin with. Finally I am definitely well past getting to the point where I think overriding rebase as a default of pull is more hassle than its worth. Especially when people merge a support branch and then pull again, resulting in both the merge being undone and a duplicate commit being made. It doesn't help that Git does all this without a single warning or message.
I guess what I've leant is that some developers just want to get their work done in peace. They don't really care about the finer points of source control, and they especially don't care about indexes, rebasing, DAGs or any other such nonsense. Perhaps in this respect Mercurial might have been a better choice? Less sharp edges for people to run into, and I could always run hg-git locally to keep me happy. Who knows at this point. The decision has been made; the ship most definitely sailed. On the upside, I'm fairly sure most, if not all, people are happy about the switch away from Clearcase. The world is a better place for it too.
My next obsession? Scala!
The good news is: we won!
That is to say that we now use Git for a majority of projects at work, with the last few being migrated as we speak. Huzzah!
How did it happen?
Sadly my persistent nagging made very little difference in the end. Fortunately, however, money did; the cost of continued licensing of Clearcase was not attractive to the bean counters upstairs. Combined with the increasing globalisation of our team meant that Git, or at least a free DVCS, became a very real option (free being the operative word). Mercurial was the first choice, but by those who didn't have any hands-on experience with either; a mistake I have blogged about earlier. Given that my team and I had been using Git in-anger for at least 6 months we fortunately (or unfortunately) managed to convince them to think again.
We decided to host our source in The Cloud; specifically on GitHub. Now GitHub is an impressive site and I use it for all my open source projects. However, I do have one minor gripe: sadly GitHub has not yet implemented fine-grained security for Organisations. Given that the majority of users are open source and would most likely fork-on-write this makes sense. In our environment where we effectively work on a single repository it would be nicer if we could delegate a project leader or leaders and have them add/remove users for their team. Add the moment only 'Owners' can appear to do that, who can then also see billing information. The other problem is that with push authorisation you get force-push as well, which can be extremely dangerous in the hands of some. Not an ideal situation to be sure. Before the official migration we used the awesome gitolite which handled security beautifully, even down the branch/force push level. I would hope that at some point GitHub will adopt something more like that; but for now things are still 1 millions time better than using you-know-what and we can't complain too much.
I should add that the obvious rebuttal to this complaint is that we probably need to consider switching to a more distributed/pull-model of collaboration, where everyone has their own repository. I don't disagree with this necessarily, but having a push model gives us high visibility of changes and a quick development cycle. A discussion for another day perhaps.
And how about the users? Are they happy?
As much as I love Git, I'm not finding it any easier to teach some of the finer points of Git and DVCS, and people still ask me for help on a daily basis. It does take a while for the difference between checkout and reset to sink-in, which I can emphasise with. Also multiple branches from within one repositories is something that causes confusion to begin with. Finally I am definitely well past getting to the point where I think overriding rebase as a default of pull is more hassle than its worth. Especially when people merge a support branch and then pull again, resulting in both the merge being undone and a duplicate commit being made. It doesn't help that Git does all this without a single warning or message.
I guess what I've leant is that some developers just want to get their work done in peace. They don't really care about the finer points of source control, and they especially don't care about indexes, rebasing, DAGs or any other such nonsense. Perhaps in this respect Mercurial might have been a better choice? Less sharp edges for people to run into, and I could always run hg-git locally to keep me happy. Who knows at this point. The decision has been made; the ship most definitely sailed. On the upside, I'm fairly sure most, if not all, people are happy about the switch away from Clearcase. The world is a better place for it too.
My next obsession? Scala!
ClearCase strikes back July 12, 2009 at 01:57 AM | categories: svn, git, java, python, dvcs, linux, clearcase | View Comments
So it's been almost a year since my last post and I obviously haven't had much
to say. I'm back to talk about the exact same thing, again. I really need to
find something other than version control to obsess about...
Recently I made the switch to Ubuntu at work. This left me without ClearCase, as it requires some derivative of enterprise Redhat or SUSE, which didn't interest me in the slightest. I can only assume this is partially due to the complexities of managing a separate binary kernel module required by MVFS, their virtual filesystem. As an alternative Rational seem to be pushing their Java implementation ClearCase Remote Client (CCRC). CCRC is essentially an Eclipse plugin communicating to a server - via a mixture of Web Services and RPC, which in turn has the 'real' client installed. As you would expect this seems to have some performance trade-offs, and it also feels to me quite unstable at the moment, often throwing strange errors unrelated to ClearCase. It's certainly better than nothing though.
This, of course, raised immediate problems for me as I refuse to go back to using ClearCase directly now that I've tasted the good (and fast) Git life. Because I obviously can't seem to help myself I ported the gitcc to Java to use their new libraries, although it's definitely the last time.
One good thing to come out of this new version is the cross-platform ability to login as any user, bypassing the domain-specific authentication that is normally used. At least in my situation ClearCase was authenticated via our Windows domain and I could see no quick and safe way to switch between users. This was/is essential for us to preserve the identity of the author of each commit in ClearCase. I strongly suspect there is a way around this, but for the life of me I couldn't think of any. The one alternative that occured to me would have been to run gitcc on something other than Windows configured with NFS and 'sudo su' to each user as required. For reasons I am unable to articulate I wasn't all that happy with the idea and never bothered.
I've been running a gitcc daemon at work for the past week with great success. It's funny how the speed difference of pushing/pulling affects your work habits. Before I used to watch the checkin and rebase console output, like being glued to TV. Especially because we're using UCM this could take up to a few minutes to do it's rebase/deliver business, which would distract me and I would often 'hold off' on pushing or pulling because I wanted to do it all in one hit to save time. I no longer have that problem, being able to 'fetch' whenever I like, knowing it only takes (literally) a second to complete. Of course in the background the daemon is chugging away, syncing with ClearCase at its own slow-and-steady pace.
Git seems to be finally taking off at work, after a slow and bumpy start. Personally I'm thrilled to be using Git at work regardless of what anyone else is doing. However some part of me isn't satisfied with just stopping there. It pains me to see others at work using a sub-par tool when I know there is a much better one available. The worst part is that most of them don't even know what they're missing, having never used anything else. Having originally come from SVN I couldn't bear to have to checkout files every time I wanted to make a change and I knew it didn't have to be that way. A co-worker even commented that they 'like' knowing when they're modifying files just in case they accidentally typed something without realising it. That may be true, but I strongly suspect after using Git for a few days they would be unlikely to revert. That's not even taking into consideration the many other benefits of using a DVCS - like local branching.
What remains is the biggest hurdle: taking it to the 'next level'. A handful of developers are currently using it individually, but now that we have daemon available we can start to look at essentially ditching ClearCase, at least for our team. Part of the problem with working for a 'largish' company is their slow and cautious approach when introducing new technologies. From their perspective they have to consider what happens if I get run over by a bus tomorrow (those damn buses - always threatening to run us over). No one else really has the same intimate knowledge of gitcc and its inner workings. How much time and money would be wasted having to fix it, or revert to ClearCase if I was no longer there? The other issue is one of training and expertise. Now you have the problem of supporting two completely different source control systems, including training for, and maintenance of, both.
The final problem that I face is one of Git's maturity. I love Git. Without a doubt it is the most advanced version control system available today. Sure it's got more than its fair share of warts, but I wouldn't use anything else (yes I know about Mercurial, and in any other universe without Git I would be using it quite happily). Unfortunately what it lacks at the moment is all that boring Windows GUI and administration tools that managers and non-technical people need (and I don't just mean TortoiseGit). I'm sure they'll be here soon enough, but in the meantime I would still hesitate if asked 'is Git ready'. The problem is that the viable alternative, at this very second in time, would most likely be SVN, which isn't a bad system, but when compared to any modern DVCS it looks increasingly archaic. If we switched to SVN now I strongly suspect there would be pressure to switch (again) to something more powerful in the not-too-distant future.
I'd love to use Git at work as our sole version control. Unfortunately as a developer I don't really have the time or patience to play politics and ensure that all aspects of a migration are planned for. However, it seems silly to me that if a majority of your developers start to use an alternative system for any aspect of work then there isn't some reflection on your current one and how you can make your working environment that much more productive. I'm not suggesting for a second that we replace ClearCase tomorrow. Instead I am simply proposing a gradual adoption, or at least trial, of Git over ClearCase. Hell, we're pretty much doing that now anyway, just not officially.
Finally, the other revelation to come out of this is how truly entrenched in Java I have become. I found with the Python implementation I would often worry about making changes because I would have to spend part of the time investigating how to implement something 'in Python' rather than just being able to focus on the problem at hand. This has nothing to do with Python, it still remains a fantastic language, but perhaps says something about me getting older and less adaptable when learning new things. I hope I never reach that point where I'm too afraid to try something new, although I'm certainly becoming far less likely to (at least seriously) learn a new language in my spare time. I'm just hoping one of these days I get to use Haskell at work. :)
Recently I made the switch to Ubuntu at work. This left me without ClearCase, as it requires some derivative of enterprise Redhat or SUSE, which didn't interest me in the slightest. I can only assume this is partially due to the complexities of managing a separate binary kernel module required by MVFS, their virtual filesystem. As an alternative Rational seem to be pushing their Java implementation ClearCase Remote Client (CCRC). CCRC is essentially an Eclipse plugin communicating to a server - via a mixture of Web Services and RPC, which in turn has the 'real' client installed. As you would expect this seems to have some performance trade-offs, and it also feels to me quite unstable at the moment, often throwing strange errors unrelated to ClearCase. It's certainly better than nothing though.
This, of course, raised immediate problems for me as I refuse to go back to using ClearCase directly now that I've tasted the good (and fast) Git life. Because I obviously can't seem to help myself I ported the gitcc to Java to use their new libraries, although it's definitely the last time.
One good thing to come out of this new version is the cross-platform ability to login as any user, bypassing the domain-specific authentication that is normally used. At least in my situation ClearCase was authenticated via our Windows domain and I could see no quick and safe way to switch between users. This was/is essential for us to preserve the identity of the author of each commit in ClearCase. I strongly suspect there is a way around this, but for the life of me I couldn't think of any. The one alternative that occured to me would have been to run gitcc on something other than Windows configured with NFS and 'sudo su' to each user as required. For reasons I am unable to articulate I wasn't all that happy with the idea and never bothered.
I've been running a gitcc daemon at work for the past week with great success. It's funny how the speed difference of pushing/pulling affects your work habits. Before I used to watch the checkin and rebase console output, like being glued to TV. Especially because we're using UCM this could take up to a few minutes to do it's rebase/deliver business, which would distract me and I would often 'hold off' on pushing or pulling because I wanted to do it all in one hit to save time. I no longer have that problem, being able to 'fetch' whenever I like, knowing it only takes (literally) a second to complete. Of course in the background the daemon is chugging away, syncing with ClearCase at its own slow-and-steady pace.
Git seems to be finally taking off at work, after a slow and bumpy start. Personally I'm thrilled to be using Git at work regardless of what anyone else is doing. However some part of me isn't satisfied with just stopping there. It pains me to see others at work using a sub-par tool when I know there is a much better one available. The worst part is that most of them don't even know what they're missing, having never used anything else. Having originally come from SVN I couldn't bear to have to checkout files every time I wanted to make a change and I knew it didn't have to be that way. A co-worker even commented that they 'like' knowing when they're modifying files just in case they accidentally typed something without realising it. That may be true, but I strongly suspect after using Git for a few days they would be unlikely to revert. That's not even taking into consideration the many other benefits of using a DVCS - like local branching.
What remains is the biggest hurdle: taking it to the 'next level'. A handful of developers are currently using it individually, but now that we have daemon available we can start to look at essentially ditching ClearCase, at least for our team. Part of the problem with working for a 'largish' company is their slow and cautious approach when introducing new technologies. From their perspective they have to consider what happens if I get run over by a bus tomorrow (those damn buses - always threatening to run us over). No one else really has the same intimate knowledge of gitcc and its inner workings. How much time and money would be wasted having to fix it, or revert to ClearCase if I was no longer there? The other issue is one of training and expertise. Now you have the problem of supporting two completely different source control systems, including training for, and maintenance of, both.
The final problem that I face is one of Git's maturity. I love Git. Without a doubt it is the most advanced version control system available today. Sure it's got more than its fair share of warts, but I wouldn't use anything else (yes I know about Mercurial, and in any other universe without Git I would be using it quite happily). Unfortunately what it lacks at the moment is all that boring Windows GUI and administration tools that managers and non-technical people need (and I don't just mean TortoiseGit). I'm sure they'll be here soon enough, but in the meantime I would still hesitate if asked 'is Git ready'. The problem is that the viable alternative, at this very second in time, would most likely be SVN, which isn't a bad system, but when compared to any modern DVCS it looks increasingly archaic. If we switched to SVN now I strongly suspect there would be pressure to switch (again) to something more powerful in the not-too-distant future.
I'd love to use Git at work as our sole version control. Unfortunately as a developer I don't really have the time or patience to play politics and ensure that all aspects of a migration are planned for. However, it seems silly to me that if a majority of your developers start to use an alternative system for any aspect of work then there isn't some reflection on your current one and how you can make your working environment that much more productive. I'm not suggesting for a second that we replace ClearCase tomorrow. Instead I am simply proposing a gradual adoption, or at least trial, of Git over ClearCase. Hell, we're pretty much doing that now anyway, just not officially.
Finally, the other revelation to come out of this is how truly entrenched in Java I have become. I found with the Python implementation I would often worry about making changes because I would have to spend part of the time investigating how to implement something 'in Python' rather than just being able to focus on the problem at hand. This has nothing to do with Python, it still remains a fantastic language, but perhaps says something about me getting older and less adaptable when learning new things. I hope I never reach that point where I'm too afraid to try something new, although I'm certainly becoming far less likely to (at least seriously) learn a new language in my spare time. I'm just hoping one of these days I get to use Haskell at work. :)
A Clearcase for Git October 25, 2008 at 07:30 PM | categories: python, clearcase, git | View Comments
My introduction to Clearcase began about two years again at my now current
current job. Coming from SVN it seemed, and still is, so overly complicated and
slow. What is this "config spec" you speak of? Anyway, I could write essays
devoted to my loathing of Clearcase but this is not what this is about.
I've been tempted for quite some time to introduce a DVCS, any DVCS, into the mix to make my life easier. However, I was concerned it may end up being just as much work as just sticking with Clearcase. I certainly didn't want to have to stuff around with manually deleting and adding files and native moving/renaming was also essential.
I found some helpful scripts, but nothing that really just worked.
Anyway, I finally got my act together and put together a simple bridge which can push and pull changes into Git. About bloody time!
http://github.com/charleso/git-cc/tree/master
The README is probably a good place to start for anyone interested.
Please note: It's still very much a work in progress. However, I'm already using it at work to much joy and relief. Not more pointless checking out on private branches. No more VOB locking or stupid branch renaming for releases. Just sweet, sweet Gitness.
A few basic examples:
Finally, it was so refreshing to finally work with Python again. I'd forgotten how simple and concise it is to use. I'm so entrenched in Java that, like an addiction, it's too easy to fall back on. That's not to say I'm addicted to Java, nor do I hate it particularly, but it's just so comfortable and I just don't have to think about it anymore. Sad really.
I've been tempted for quite some time to introduce a DVCS, any DVCS, into the mix to make my life easier. However, I was concerned it may end up being just as much work as just sticking with Clearcase. I certainly didn't want to have to stuff around with manually deleting and adding files and native moving/renaming was also essential.
I found some helpful scripts, but nothing that really just worked.
Anyway, I finally got my act together and put together a simple bridge which can push and pull changes into Git. About bloody time!
http://github.com/charleso/git-cc/tree/master
The README is probably a good place to start for anyone interested.
Please note: It's still very much a work in progress. However, I'm already using it at work to much joy and relief. Not more pointless checking out on private branches. No more VOB locking or stupid branch renaming for releases. Just sweet, sweet Gitness.
A few basic examples:
python gitcc.py init
d:\views\your_clearcase_view
python gitcc.py rebase
# Obligatory comment about getting coffee and doing work
python gitcc.py checkin
That's about it really. I'm interested to see if anyone else finds this
useful or has any suggestions.
Finally, it was so refreshing to finally work with Python again. I'd forgotten how simple and concise it is to use. I'm so entrenched in Java that, like an addiction, it's too easy to fall back on. That's not to say I'm addicted to Java, nor do I hate it particularly, but it's just so comfortable and I just don't have to think about it anymore. Sad really.