PDA

View Full Version : Examples not part of the Automake build



bvanevery
03-19-2013, 09:52 PM
In 2.3.x and 3.0 on Linux, I notice that Examples are not built as part of the build. Also there's no default top level structure to build all of the Examples if in fact you want them. Pretty much this part of the build is commented out, although it could be made active and clearly was active at some point in the past. I can build individual Examples by going to the specific directories, opening a console window, and typing "make". Some work, others don't. I think from code exercise standpoint, it would be better to have these things built regularly. It's a form of testing and it's useful to users to see how things work.

Speaking of Testing, the same is pretty much true of the Tests directory. The Makefile targets are hugely out of date, bearing no relationship to the subdirectories actually present in Tests. I can go to individual sub-subdirectories and type "make" but things don't work due to PKG_CONFIG_PATH errors. Perhaps it expects ClanLib stuff to already be installed before tests can be done. I don't think that's an ideal way to test, but I suppose that's a religious issue.

I notice that Visual Studio has .sln files for both Examples and Tests. This sort of thing is motivation for a cross-platform build like CMake or Premake.

Judas
03-20-2013, 01:59 AM
The Tests directory doesn't contain unit tests. It is a folder of various random projects used during development of modules. They aren't built because there is no guarantee they still compile after they served their original purpose.

As for real tests we don't use that. ClanLib is a hobby project and at least personally I would never write an unit test voluntarily. Only when paid (and explicitly required to) do I do stuff so mindbendingly boring. :D

rombust
03-20-2013, 07:40 AM
However, I have created various unit tests for clanCore, most of them are in Tests/Core.

There are a lot of tests in Tests that are very out of date, and some are now obsolete and should be deleted.

Almost all examples now work on Windows, there are now only a few that still require porting from ClanLib 2.3

I was in the process of fixing all the Examples and Tests, but development is currently on hold, whilst I finish other (non clanlib) projects.

ClanLib 0.8 used to automatically build the examples. (via make examples). That was disabled by myself (many years ago) because I didn't use it, and it was incomplete.
For example, if "./configure --disable-clanGUI" is used, then you do not want examples that use the gui to be compiled.


I can go to individual sub-subdirectories and type "make" but things don't work due to PKG_CONFIG_PATH errors. Perhaps it expects ClanLib stuff to already be installed before tests can be done
All the examples and tests on linux expect clanlib to already be installed. This is probably wrong.

Judas
03-20-2013, 04:19 PM
I don't mind the Tests directory only containing actual unit tests. All I'm saying is that I'm not writing anyone myself on a hobby basis. :)

However if we do want those tests to be compiled and run as part of the build system, then we probably need a new directory (or a specific subdirectory) for the kinds of test projects the folder originally was made for.

bvanevery
03-20-2013, 05:17 PM
The Tests directory doesn't contain unit tests. It is a folder of various random projects used during development of modules. They aren't built because there is no guarantee they still compile after they served their original purpose.

Finally read the README.txt in that directory, which does indeed say what you just said. Still, this kind of structure and methodology is contrary to open source expectations. I wish I could think of a more appropriate directory name, like "chance," "junk," "wildcard," or "attic," but I don't think there's any generally recognized convention for what you're doing. Instead, I'm going to try to convince you not to do it. :eek:


As for real tests we don't use that. ClanLib is a hobby project and at least personally I would never write an unit test voluntarily. Only when paid (and explicitly required to) do I do stuff so mindbendingly boring. :D

Well, your candor is definitely useful as I try to evaluate whether potential military clients will ever make use of ClanLib. This would seem to put things in the "less likely" column, along with build infrastructure and choice of code repository (CMake, Mercurial) compared to Ogre. But you guys have been pretty responsive to actually fixing build issues, which has kept me plugging along. I would point out that "no no no this is my hobby" is not really the way to grow a library in open source. It depends on your goals, whether robustness and wider adoption are part of your goals. You say unpaid unit tests are anathema to you. Well, how do you think I feel about debugging build systems for open source projects that I'm not even sure are useful to me yet? I don't tell other open source people I won't do it. Rather, I chug through the code and deal with the bugs out of discipline, for $0, because that's the only way open source functions effectively. I do it because I think the potential value proposition of a "shared commons" is there. If I stop believing that, then I will write my own 3D engine. Which I can do, I have the technical background for it, and plenty of starter / example code to draw on. I'm just hoping that either ClanLib or Ogre might actually be suitable to my purposes so I can do less work. I've also fired up Irrlicht; they claim OpenGL 3.x support but I don't think they really have it. You guys seem to actually have something for 3.x which is why I'm still here.

So I would suggest, if it's a lightweight undertaking, that you just make whatever is in the source tree actually build and work on a regular basis? Why ship code that doesn't work and is publicly declared to have a low chance of working? It doesn't reflect well on your project. I'd suggest just keeping that code in some other repository that no one sees, if you really don't want to invest any effort into having it work. The point of open source is to get other people to invest such effort though. You get that effort by committing to having things work, so other open source people don't feel it's a waste of time to fix.

rombust
03-20-2013, 11:28 PM
It's very refreshing when someone like yourself posts an honest opinion about open source :)

All the ClanLib developers have differing opinions of what they want from ClanLib. I think that's how it's surviving after 15 years of development.

4 years ago, I was very keen to get ClanLib used far and wide, advertising it etc. It worked well.

I used ClanLib 2.2 for commercial use around year 2010, and ported the codebase to 2.3 the following year.

Back then, ClanLib 2.3 was amazing, it was stable, flexible and portable.

The failing with ClanLib 2.3 was its reliance of OpenGL. If would not scale to Direct3D and there was a namespace conflict with OpenCL.

So the decision was to create a new branch ClanLib 3.0.
It now uses the latest C++11 standard.
clanDisplay had to split the GraphicContext into 2 classes (Canvas and GraphicContext), to handle 3D environment more effectively.
clanGUI was getting dated, it was difficult to create a dynamic gui, that repositioned buttons depending on the window size. We now have a full CSS2 and partly CSS3 parser that it uses.

At stated, ClanLib 3.0 is new, and needs fine tuning to make it into a tidy package.

The only reason I develop on ClanLib is because of it's free licence (not LGPL'd). I can extract parts I want, for my own work. And because i'm kind, I put something back into it (in the hope that others will do the same, so I can loot their code)

Now, I don't care who uses it. I have become very disillusioned with the open source ideologies.

Since ClanLib will never give me fame, fortune and dancing girls, I only develop parts that interest myself.

I do foresee though, by the end of this year, ClanLib 3.0 will be at a state where it can be released as a complete package (stable API and all examples working). But since ClanLib 2.3 fits my personal needs, there is no rush from myself. But in a couple of years time, ClanLib 3.0 might be required, so since I know the library intimately, it's in my best interest that it's stable.

(Note, I have been developing Open Source since around 1998). (Coding too many years before that)

Btw, ClanLib 3.0 supports up to OpenGL 4.2

Judas
03-21-2013, 06:17 AM
Finally read the README.txt in that directory, which does indeed say what you just said. Still, this kind of structure and methodology is contrary to open source expectations. I wish I could think of a more appropriate directory name, like "chance," "junk," "wildcard," or "attic," but I don't think there's any generally recognized convention for what you're doing. Instead, I'm going to try to convince you not to do it. :eek:


Ah, but I would argue that about 95% of all unit tests I've ever seen also match the category "chance" pretty well. The amount of work required to write a truly 100% coverage unit test requires so much resources it is really only justifiable in a few select industries (aircrafts, medicine, etc).

Pretty much everywhere else the unit tests just tests a few select use cases that the coder could be bothered to code, and then everyone goes into *lalala land* pretending it is now a solid test.

The way I see it the only advantage real unit tests have is that you can run them again if you change the code, and then if the few select code paths actually tested break it will be noticed. Now you could argue this is a rather important advantage, and I don't disagree that it is an important feature, especially if code changes a lot. But most people doesn't seem to acknowledge that all anti-bug measures you take is a trade-off between bugs found/avoided versus extra time spent. Also keep in mind that unit tests have a big weakness when it comes to testing systems depending on other systems, i.e. testing a class like TCPListen or TCPConnection gets complicated fast (if its going to have any realistic coverage of actual usage code paths).

So my "chance" test projects finds about 90% the same errors as the typical unit test out in the wild, except that they aren't maintained and continously run. The time to code them is significantly less, which means I can spend my time doing other things instead. When you have limited resources available something has to give. Likewise I only update them if I have reason to believe something broke which relates to the test. I don't have luxery of being able to hire testers and additional coders for my hobby.


Well, your candor is definitely useful as I try to evaluate whether potential military clients will ever make use of ClanLib. This would seem to put things in the "less likely" column, along with build infrastructure and choice of code repository (CMake, Mercurial) compared to Ogre. But you guys have been pretty responsive to actually fixing build issues, which has kept me plugging along. I would point out that "no no no this is my hobby" is not really the way to grow a library in open source. It depends on your goals, whether robustness and wider adoption are part of your goals. You say unpaid unit tests are anathema to you. Well, how do you think I feel about debugging build systems for open source projects that I'm not even sure are useful to me yet? I don't tell other open source people I won't do it. Rather, I chug through the code and deal with the bugs out of discipline, for $0, because that's the only way open source functions effectively. I do it because I think the potential value proposition of a "shared commons" is there. If I stop believing that, then I will write my own 3D engine.

I know exactly how it feels like to use someone elses build system. For example, I literally spent about 8 hours with the v8 javascript engine's build system to finally get it to generate .lib and .pdb files with the settings I needed. What do you think would happen if I wrote to Google that they should ditch their custom-written python+cygwin Android build system in favor of CMake, or and that they should switch from their git+svn setup to Mercurial instead?

Or how about Mozilla that also needs cygwin and insists that the build system needs to be in C:\Mozilla? I never actually managed to build their TLS crypto library with the settings I needed and had to change my entire installation process to use their library.

Just to make it completely clear, I am not against properly written unit tests. Especially not for the math parts where errors otherwise can be very difficult to track down. Or to have something else than autohell as the unix build system. But it is just not in my interest to contribute to such systems at current time. I am using the unix build system that Rombust has so generously maintained over the years, and until someone else replaces it with something better, this is probably how it will stay.


So I would suggest, if it's a lightweight undertaking, that you just make whatever is in the source tree actually build and work on a regular basis? Why ship code that doesn't work and is publicly declared to have a low chance of working? It doesn't reflect well on your project. I'd suggest just keeping that code in some other repository that no one sees, if you really don't want to invest any effort into having it work. The point of open source is to get other people to invest such effort though. You get that effort by committing to having things work, so other open source people don't feel it's a waste of time to fix.

Yes, maybe we should move 3.0 into a private repository.

bvanevery
03-21-2013, 02:17 PM
Ah, but I would argue that about 95% of all unit tests I've ever seen also match the category "chance" pretty well. The amount of work required to write a truly 100% coverage unit test requires so much resources it is really only justifiable in a few select industries (aircrafts, medicine, etc).

Who says you're doing that? You've got a /tests directory with some code in it. Nobody said you are writing unit tests, or have to, or that it has to be a moon shot. I'm saying, make it build. Make it regularly exercised so it isn't "open sores" in the source tree.



I know exactly how it feels like to use someone elses build system. For example, I literally spent about 8 hours with the v8 javascript engine's build system to finally get it to generate .lib and .pdb files with the settings I needed. What do you think would happen if I wrote to Google that they should ditch their custom-written python+cygwin Android build system in favor of CMake, or and that they should switch from their git+svn setup to Mercurial instead?

I don't care what Google has to say about that. I got hired by Mozilla in 2007 to convert their Firefox build from Autoconf to CMake. Unfortunately I underbid the amount of time the project would require, and I failed. The point is, companies and organizations decide it's time to "move on," or try to, for rational reasons. It's not interesting whether Google "might resist" a different build system. They might not.


Or how about Mozilla that also needs cygwin and insists that the build system needs to be in C:\Mozilla?

It wasn't for lack of throwing some money at professional CMake development. I wrote a "throwaway" script for converting Automake to CMake for that specific project. That code still exists, but I haven't heard of anyone picking it up. It was a crapload of regular expressions. My conversion strategy was actually working, it was faster and more accurate than rewriting stuff from scratch. There was just too much code to get it done within the budget. That's a risk when contracting a big project. It's someone else's code, you don't have time to figure it all out before you start billing, so who knows what you'll run into? I feel a little bad that I couldn't bring home the bacon for them, but not horrible.

Lest it sound like I'm gunning to do a CMake build for you, I'm not. That project taught me just how much or little I should be spending time on build systems. The point is to write a game, not to be a build guy. But, of course I am CMake proficient and I know what it can or can't do as infrastructure. Ogre committed to CMake and Mercurial some time ago and there are strategic advantages to having done that. Using better tools attracts better developers and increases platform reach.


Yes, maybe we should move 3.0 into a private repository.

Up to you. That's a very bad move as far as attracting other open source developers though. Do you want to play on your own, or do you want to grow this thing and get labor? Nobody's gonna write you a build or fix your bugs if they can't even see it.

bvanevery
03-21-2013, 02:25 PM
Now, I don't care who uses it. I have become very disillusioned with the open source ideologies.

The only relevant ideology AFAIAC is having a "software commons" that is viable for commercial projects. People and their personalities are extremely irritating. The dirty little secret of open source is it's not about the code, it's about the people. What are you going to do if some lead developer for some big project turns on you? Fork it and carry all the support burdens yourself? That's typically a huge amount of work and not practical.


Since ClanLib will never give me fame, fortune and dancing girls, I only develop parts that interest myself.

You're supposed to write the next Minecraft for that, using ClanLib as infrastructure. Which means ClanLib itself is not the mission, but an important supporting part of the mission. Effort should be budgeted accordingly.

rombust
03-21-2013, 03:15 PM
I had a quick look in the tests folder.

Most of them are actually valid. The problem is there was a major breaking change between ClanLib 2.3 and 3.0 (Moved ClanLib into a namespace, and the introduction of Canvas). In time, they will be fixed (that's my personal mission :) )

Personally, I would like to see ClanLib use GIT and CMake (and i'm sure most developers would agree)

Changing to GIT from SVN is easy. The Assimp library only recently ported their source code to GIT from SVN, and it had a very positive response. Now that TortoiseGIT is usable (I believe), it shouldn't be a problem.
(We probably all need to agree though.)

Everyone wants to ditch autotools. For linux, it would be very helpful. For win32, we can have CMake running alongside configure.exe
It's a big complex job. If I remember correctly, 2 developers have tried it, and given up.

bvanevery
03-21-2013, 03:36 PM
Changing to GIT from SVN is easy. The Assimp library only recently ported their source code to GIT from SVN, and it had a very positive response. Now that TortoiseGIT is usable (I believe), it shouldn't be a problem.
(We probably all need to agree though.)

Don't be afraid of Mercurial instead. TortoiseHg is totally field proven on Windows. I used it successfully for some Wesnoth campaign development 2 years ago, with some guys who had no experience with version control. Also as a minor perk for poaching developers from other projects, Ogre is a Mercurial culture. Generally speaking, the more Windows oriented you are, the more Mercurial is a better bet. The more Linux oriented you are, Git is a better bet. I'm happy with either as I'm experienced with Mercurial but Linux is the platform I'm developing for now. TortoiseHg doesn't work on my Lubuntu LDXE desktop, but I know the command line enough to be dangerous, and I found "hgview" to give me the pretty visualizations of project branches and so forth. Which was needed, because I was using the wrong Ogre branch in my ignorance.


Everyone wants to ditch autotools. For linux, it would be very helpful. For win32, we can have CMake running alongside configure.exe
It's a big complex job. If I remember correctly, 2 developers have tried it, and given up.

Consider also Premake. It is Lua based, so you would get a better scripting language than the clunky CMake script. Premake lacks header / library interrogation features though. CMake has more of that. However, some of those interrogators are rusty, particularly the OpenGL ones, so you may end up having to write your own interrogator scripts anyways. In which case, writing them in Lua in Premake wouldn't be a disadvantage.

I can't really offer to help on build stuff until I've decided ClanLib has "enough to it" that it's the engine I'll use for my projects. You're competing with Ogre and Irrlicht. I'm still figuring out what they can or can't do as well. Ogre is CMakeified. Irrlicht has simple Makefiles and recently added .sln files, so clearly they're at a pretty early stage of build system evolution. However since my development platform is Linux, this doesn't bother me. In fact, I'd even consider being Unix-centric rather than Windows-centric an advantage. If I end up writing my own 3d engine, it'll be Linux OpenGL 3.x only, to stay focused. From bitter experience, I feel the exercise of wrapping OSes and 3D APIs is misguided. First make money on a game release, then port, I say. But if 3 projects have already done it, well, if those wrappers don't suck and have people maintaining them, that's different. Haven't even begun to evaluate the "don't suck" part yet. I know builds really well, so that's what I've been dealing with first.

Also I would strongly suggest that if you do come up with a standard cross-platform build system, you commit ideologically to getting rid of your homebrewed configure.exe. When you have developers maintaining 2 build systems side by side, what happens is one of 'em gets short shrift. When the lead developer likes the old system, he'll slack off about dealing with the new build because he'd rather stay comfortable and not have to lift fingers. The new buildmaster will eventually get mad at him because he's slacking. When that lead developer has the power to get rid of you, the new buildmaster, you've just thrown 1 man-year of your open source life down the drain. People can say they would like to have a new build system, and be open to the option of another build system, but you must make them ideologically commit to the new system. Or you will be wasting your time in Dilbert land with all those open source "personalities" I alluded to earlier. I'm sadder but wiser. This guy was obsessing about being able to run his code on every single GCC screwy embedded Unix platform ever devised, and didn't care about Visual Studio. Even though the whole point of the CMake build was to support Visual Studio, it covered all the major platforms that real developers worry about, and cross-compiling capabilities were slated to appear in CMake in 6 months or so. He got rid of CMake and Autoconf and went back to hand rolled Makefiles.

bvanevery
04-04-2013, 08:14 PM
In ClanLib 3.0 SVN rev 9678, I notice that no Makefiles are generated for the Examples subdirectories or sub-subdirectories. There are no Makefile.am files either. Makefiles should exist if someone wants to build the Examples, even if they are not built by default. For instance, rombust just asked me to build the ShaderEffect example on Linux to test it out, and I couldn't.

rombust
04-04-2013, 08:26 PM
Ah, that's because I created that makefile in "ClanLib 3.0 SVN rev 9678", you need to update :)

bvanevery
04-04-2013, 09:01 PM
Got that. Ok, looks like the sub-subdirectories all have Makefiles, so things can be built from there.