PDA

View Full Version : Including Accessibility into clanlib



jaffar
05-16-2014, 02:05 AM
Hi. My Name is Jaffar. I am a blind programmer/software developer from Singapore and I specialize in c++ and Java programming. While browsing through WikiPaedia, I came across the ClanLib game engine, and because I am now in the process of writing a game and Out of curiosity, I downloaded it, and from what I've observed after trying it, I find that it is the most intuitive of all the game engines I've tried so far. Now, as I said before, I am a blind computer programmer/user and I navigate my computer, the web and other apps through the use of a Screen reader software which, if standard texts and windows controls are present, will speak them out to me. I see a great potential in clanlib, and I am wondering if accessibility via screen readers can be implemented into ClanLib. Just to put in into perspective, I compiled and ran the Sound sample earlier. From the code, I noticed that there are buttons and a combo box. Yet my screen reader can't access those as it might in say, ms office or ms visual studio because the texts are not spoken due to them not being standard windows output. I would think that a little adjustment can be made to for example, the layout.xml resource file so that my screen reader can see the text on screen. If you guys don't mind, i'll be happy to help and take a major role in developing accessibility and implementing it into ClanLib. Thanks for a great game engine.

moogtrain
05-16-2014, 04:56 AM
Hi. Welcome to ClanLib. I'm deeply interested in the idea, but I'm not familiar with tools used by the blind for programming, and I've relied on my vision so much I can't think of a single instance where I'm use my ears as I program things. I'm sure the others here are unfamiliar with the concept as well. Can you describe more about how you would write a program and the accessbility tools that you would need on this framework?

jaffar
05-16-2014, 05:26 AM
Hi. I accomplish my programming tasks with Visual Studio, DevCPlusPlus and the Eclipse IDE. These are mostly accessible to my screen reader which interprets vocally what we type or what it "sees" on screen. To date, there are 4 major screen readers in use, Jaws For Windows which is what I'm using, www.freedomscientific.com Window Eyes, www.gwmicro.com NVDA, www.nvda-project.org and SuperNova www.airdolphin.com and these are the major screen readers used by the global blind community. So, with my screenreader and the IDES I mention above, I've been able to pursue my programming almost independently accept for when I need sighted opinion for things like appearance an layout of course, then i'll ask for my wife's view as it were. So if for example the text on a ClanLib-based button could be made to appear "normal" to a screen reader, then I would be able to interact with that button, thus allowing me to play any accessible designed game unhindered since my screen reader can tell me to say press an ok or quit button or make a choice of available devices through a combobox whose selection can be read by the screen reader. Thanks.

Judas
05-16-2014, 08:57 AM
Adding screen reader support requires supporting the UI Automation API in Windows.

The first step in doing this would be to implement the WM_GETOBJECT message in win32_window.cpp and then return a IRawElementProviderSimple interface. This is where it begins to get really tricky because the GUI in ClanLib doesn't use Win32Window directly. The GUI uses the WindowManager abstraction to create top-level windows, and the window manager in turn creates a DisplayWindow with the actual system window. DisplayWindow uses the Win32Window class internally.

Thus, for Win32Window to implement the IRawElementProviderSimple interface, it would need to somehow be able to ask the GUI about which GUIComponent objects there are and what their hierarchy is. One way of doing this would be to create some cross platform abstract class adding the same functionality as IRawElementProviderSimple, IRawElementProviderFragment and IRawElementProviderFragmentRoot. Then this interface should be assignable via a DisplayWindow::set_automation_root (automation_element) function. clanGUI would then implement this abstract class and assign one to each top-level component created.

The truly hard part of this would be to create the abstract class, since this effectively controls how much information is being sent to the screen reader. The UI Automation API defines more than 30 different control patterns that an element can support and more than 40 properties that can be retrieved. I have never used a screen reader so I can't say how many of those are truly required, but in any case such an interface would have to be designed and then supported by the standard controls in clanGUI.

jaffar
05-16-2014, 10:33 AM
Actually Screen readers do allow us to examine each window in an app so we can find out what window it is, for example if it is an edit window or a list box and whether it a child or parent window. I am now working with a GSOC student on a score writing and music composition program made with QT. Qt apps have been known to be notoriously inaccessible or bearly accessible in the past, but Accessibility has been upgraded in the latest version of qt, so I think it is quite possible to implement it into ClanLib. There are quite a few blind game developers like me across the globe, and I honestly think that the Clanlib game engine, if made to help design accessible games will stir quite an interest and will promote it's use to a wider audience, and this coming from someone who has used sdl and sfml. Cheers!