Difference between revisions of "General software issues"

From apm
Jump to: navigation, search
(Created page with "'''In a world where the digital and physical realm starts to blend''' that is physical products become metworked live acutateable and reconfigurable '''software architecture ...")
 
 
(55 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
'''In a world where the digital and physical realm starts to blend'''
 
'''In a world where the digital and physical realm starts to blend'''
−
that is physical products become metworked live acutateable and reconfigurable  
+
that is physical products become networked live acutateable and reconfigurable  
 
'''software architecture / organisation  / design or however one may call it becomes even more important''' than it is already today.
 
'''software architecture / organisation  / design or however one may call it becomes even more important''' than it is already today.
  
 
Key issues are:
 
Key issues are:
 
* stability (at best error-proofness) [http://en.wikipedia.org/wiki/Correctness_%28computer_science%29 correctness]; [http://www.haskell.org/haskellwiki/Research_papers/Testing_and_correctness research in Haskell]
 
* stability (at best error-proofness) [http://en.wikipedia.org/wiki/Correctness_%28computer_science%29 correctness]; [http://www.haskell.org/haskellwiki/Research_papers/Testing_and_correctness research in Haskell]
−
* maintainambility
+
* maintainability
−
* extendability
+
* modularity / extendability / scalability
−
* modularity
+
 
* diversity (as options for unexpected dead end routes)
 
* diversity (as options for unexpected dead end routes)
 
* optimized-specialisation conserving functionality-expanding-generalisation and vice versa
 
* optimized-specialisation conserving functionality-expanding-generalisation and vice versa
 
* highly complex version management ([http://en.wikipedia.org/wiki/Dependency_hell dependency hell])
 
* highly complex version management ([http://en.wikipedia.org/wiki/Dependency_hell dependency hell])
 +
* [[progressive disclosure]] human computer interfaces [https://en.wikipedia.org/wiki/Progressive_disclosure]
 
* ...
 
* ...
 +
 +
= Important challenges =
  
 
== File systems ==
 
== File systems ==
Line 32: Line 34:
  
 
* A good writeup about the problem and solution approaches can be found here: [http://c2.com/cgi/wiki?FileSystemAlternatives file system alternatives]
 
* A good writeup about the problem and solution approaches can be found here: [http://c2.com/cgi/wiki?FileSystemAlternatives file system alternatives]
 +
* Another pretty good writeup of why file systems in tree structure are fundamentally flawed: [https://rffr.de/wp-content/uploads/manuell/file_systems_usability.pdf "File Systems and Usability – the Missing Link" by Robert Freund]
 +
* [''todo:'' add link to NHFS paper]
 +
 +
Google once made a move in this direction.
 +
Google drive allowed (for some time) to create folder structures as an directed acyclic graph.
 +
 +
----
 +
Graph based file systems would have some similarities too wikis.
 +
Both allow one to explain the same thing in different contexts.
 +
Compared to file systems Wikis allow one to add elaborate descriptions and documentation instead of just short folder names.
 +
Wikis are not built to replace file systems though.
 +
 +
* broken links can be reconstructed
 +
* emerging concepts can be auto-detected
 +
* criterions can be combined by disjunctive and conjunctive normal form (methods from digital logic)
 +
* there is a partial order - dividing the whole graph from a chosen criterion into three parts (analog to the concurrency cone)
 +
 +
=== Possible graphical representations ===
 +
 +
* super boxes connected with directed arrows to sub-boxes or sub-boxes in super-boxes (mutually convertible). The program yEd gives a niche example of how something like that could look.
 +
* In the case of sub-boxes in super-boxes the metaphor "depth of a topic" could be taken literally for a 3D-representation.
 +
* beside the well known child tree in a graph file system an ancestor tree can also be shown
 +
* like in conventional tree style file-systems the state of all folders collapsed by default and only a view "open" is a necessity. Additionally hiding stretches of overly long paths and same level "sibling" folders might be desirable.
 +
Both situations call for remedy though which is easier in a graph than a tree.
 +
Progressive exposure like this breaks the long-standing user developer barrier making users learn by accident and making them become developers by accident.
 +
Current mainstream graphical programming languages often suffer from disregarding the importance of hiding subsystem complexity leading to horribly unmaintainable circuit krakens.
  
 
== Dependency hell ==
 
== Dependency hell ==
Line 41: Line 69:
 
* [http://en.wikipedia.org/wiki/Nix_package_manager Packet manager of experimental NixOS] - [http://www.linuxplanet.com/linuxplanet/reviews/6654/1 review]
 
* [http://en.wikipedia.org/wiki/Nix_package_manager Packet manager of experimental NixOS] - [http://www.linuxplanet.com/linuxplanet/reviews/6654/1 review]
 
* [http://en.wikipedia.org/wiki/Zero_Install multi-platform reversible java package management system]
 
* [http://en.wikipedia.org/wiki/Zero_Install multi-platform reversible java package management system]
 +
 +
'''Legacy barriers:'''
 +
 +
A high level (math based) abstraction layer is absolute essential for isolation of most of the
 +
plumbing (that is pattern matching parsing and serialization) in software.
 +
Keeping data immutable on the scale of whole operating systems and and even the whole word wide web is a necessary requirement for that.
 +
 +
Things that don't get referenced from anywhere anymore can be removed by a worldwide garbage collector.
 +
 +
* RAM to HDD
 +
* application to application
 +
* computer to computer
 +
* file to file
 +
 +
An approach to implementation is "unison" (Link: [http://unisonweb.org/]).
 +
 +
= Software for 3D-modelling and beyond =
 +
 +
{{todo| expand this chapter}}
 +
 +
Atomistic modelling: From Tom's machine phase blog comes an example of the usage of Nanoengineer-1:
 +
[http://machine-phase.blogspot.co.at/2008/04/afm-images-of-ne1-designed-origami.html DNA origami: from design to product]
 +
 +
See: [[Data decompression chain]]
 +
 +
See: [[Visually augmented purely functional programming]]
 +
 +
Bad software design may undermine the "[[Disaster proof]]" property of globally used APM.
  
 
== Further related information ==
 
== Further related information ==
  
 
Avoiding hidden state (a potential source of errors) can be compatible with interactive environments:
 
Avoiding hidden state (a potential source of errors) can be compatible with interactive environments:
 +
* Tangible functional programming [https://www.youtube.com/watch?v=faJ8N0giqzw YT Video] ([https://wiki.haskell.org/Eros Eros demo program]; TVs ... [https://wiki.haskell.org/Tangible_Value Tangible Variables])
 
* Wikipedia: [http://en.wikipedia.org/wiki/Functional_reactive_programming Functional reactive programming]; [http://conal.net/papers/push-pull-frp/ research of Conal Elliott]
 
* Wikipedia: [http://en.wikipedia.org/wiki/Functional_reactive_programming Functional reactive programming]; [http://conal.net/papers/push-pull-frp/ research of Conal Elliott]
 
* [http://en.wikipedia.org/wiki/House_%28operating_system%29 House - functional operating system]
 
* [http://en.wikipedia.org/wiki/House_%28operating_system%29 House - functional operating system]
 +
* [[reversible data processing]]
 +
* Replacing softwary with "hardcoded hardware" to make to make things that are not supposed to happen impossible. [[Self limitation for safety]]
 +
* instability of high level operating systems ...
 +
* [[Debugging]]
 +
 +
= Relation of AP Technology to new computing paradigms =
 +
 +
Related: [[Relations of APM to purely functional programming]]
 +
 +
== What is likely to be a necessity ==
 +
 +
* reversible computing
 +
Deleting data produces heat proportional to the operation temperature. In super high density computing this needs to be avoided.
 +
 +
[https://en.wikipedia.org/wiki/Janus_%28programming_language%29#Time-reversible_Computing time-reversible computing low level programming language] <br> High level programming language that match reversible computing best are functional programming languages like Liasp and Haskell.
 +
 +
== What is absolutely not a necessity but could boost development speed ==
 +
 +
Widely known:
 +
* quantum computing (depends on reversible design)
 +
* neural network compution (deep learning)
 +
 +
Barely known:
 +
* memcomputers (interspersed comuting and memory can increase performance for certain algorithms) <br> [http://advances.sciencemag.org/content/1/6/e1500031 Memcomputing NP-complete problems in polynomial time using polynomial resources and collective states]
 +
* lambda machines / graph reduction machines ([https://en.wikipedia.org/wiki/Graph_reduction_machine wikipedia]) (historic alternative to Von Neumann architecture computers) <br> Research 2015: '''Reduceron''' [https://www.cs.york.ac.uk/fp/reduceron/ webpage] [https://github.com/tommythorn/Reduceron github] implemented on FPGAs
 +
 +
Also maybe of interest:
 +
* ternary (mechanical) logic - higher radix economy but worse in transporting carries - ([https://en.wikipedia.org/wiki/Three-valued_logic wikipedia])
 +
* digital usage of analog mechanical computing mechanisms (a few bit at a time) - [http://maritime.org/doc/firecontrol/parte.htm link]
 +
* usage of p-adic arithmetic in processors
 +
 +
=== Applications ===
 +
 +
Alternative computing architectures that boost certain types of problems can often be used for optimization:
 +
* circutry; layout;
 +
* optimal molecular topologies for bigger structural [[Diamondoid molecular element|crystolecules]] e.g. brackets (aka Kaehler-brackets); ...
 +
 +
= The various obstructing rifts in Software =
 +
 +
* ...
 +
 +
= Related =
 +
 +
* '''[[Software]]'''
 +
* '''[[Gaps in software]]'''
 +
* '''[[Software trouble pages]]'''
 +
* [[Relations of APM to purely functional programming]]
 +
* [[Data decompression chain]]
 +
* [[Programming languages]]
 +
* [[Multi criterion file system]]
 +
* [[Visually augmented purely functional programming]]
 +
----
 +
* [[Gaps in software]]
 +
* [[Progressive disclosure]]
 +
----
 +
* [[Annotated lambda diagrams]]
 +
* [[Annotated lambda diagram mockups]]
 +
----
 +
* [[Bridging the gaps]] in APM development – gaps in software are mentioned last
 +
----
 +
* [[Diffing]], [[Content addressed]]
 +
 +
= External Links =
 +
 +
* [http://c2.com/cgi/wiki?FileSystemAlternatives Alternatives for the seriously flawed hierarchical (tree style) file systems]
 +
* [http://www7.scu.edu.au/1865/com1865.htm WebOFDAV — navigating and visualizing the Web on-line with animated context swapping]
 +
Wikipedia:
 +
* [https://en.wikipedia.org/wiki/Edit_decision_list Edit decision list]
 +
* [https://en.wikipedia.org/wiki/Abstraction_layer Abstraction layer]
 +
 +
[[Category:Information]]
 +
[[Category:General]]
 +
[[Category:Programming]]
 +
[[Category:Software]]

Latest revision as of 09:27, 5 May 2024

In a world where the digital and physical realm starts to blend that is physical products become networked live acutateable and reconfigurable software architecture / organisation / design or however one may call it becomes even more important than it is already today.

Key issues are:

  • stability (at best error-proofness) correctness; research in Haskell
  • maintainability
  • modularity / extendability / scalability
  • diversity (as options for unexpected dead end routes)
  • optimized-specialisation conserving functionality-expanding-generalisation and vice versa
  • highly complex version management (dependency hell)
  • progressive disclosure human computer interfaces [1]
  • ...

Important challenges

File systems

Currently used tree based and machine local file systems have their limits. Sorting the same data after multiple hierarchical criteria is impossible. To give an example: Assume one owns a lot of image-file text-file sets. The small text files are of high importance (e.g. source code) while the huge images are of relatively low importance (e.g. rendered from source). Archiving different backup levels (location, redundancy level) for file types of different importance isn't possible with the basic functionalities of tree based file systems.

File system indexing usage of meta-data and file tagging only mend but do not solve the problem. Some kind of graph based file systems are needed. Graph databases (like Neo4j?) are interesting but are only crutches if implemented on top of tree based file systems.

The limitation to serial access to mass storage disk space (now changing with random access SD drives) led to the fact that current systems are still hardware near programmed and suffer from a lack of abstraction. Data must be manually serialized for persistent storage. Net based services like Google drive and Facebook already emulate this behavior. Another interesting "on top" approach is yesod with its integrated persistence.

Google once made a move in this direction. Google drive allowed (for some time) to create folder structures as an directed acyclic graph.

Graph based file systems would have some similarities too wikis. Both allow one to explain the same thing in different contexts. Compared to file systems Wikis allow one to add elaborate descriptions and documentation instead of just short folder names. Wikis are not built to replace file systems though.

  • broken links can be reconstructed
  • emerging concepts can be auto-detected
  • criterions can be combined by disjunctive and conjunctive normal form (methods from digital logic)
  • there is a partial order - dividing the whole graph from a chosen criterion into three parts (analog to the concurrency cone)

Possible graphical representations

  • super boxes connected with directed arrows to sub-boxes or sub-boxes in super-boxes (mutually convertible). The program yEd gives a niche example of how something like that could look.
  • In the case of sub-boxes in super-boxes the metaphor "depth of a topic" could be taken literally for a 3D-representation.
  • beside the well known child tree in a graph file system an ancestor tree can also be shown
  • like in conventional tree style file-systems the state of all folders collapsed by default and only a view "open" is a necessity. Additionally hiding stretches of overly long paths and same level "sibling" folders might be desirable.

Both situations call for remedy though which is easier in a graph than a tree. Progressive exposure like this breaks the long-standing user developer barrier making users learn by accident and making them become developers by accident. Current mainstream graphical programming languages often suffer from disregarding the importance of hiding subsystem complexity leading to horribly unmaintainable circuit krakens.

Dependency hell

Current packet management systems suffer from the dreaded problem of dependency hell. Especially developers who install a lot of software packages in parallel are affected. Solution approaches include:

Legacy barriers:

A high level (math based) abstraction layer is absolute essential for isolation of most of the plumbing (that is pattern matching parsing and serialization) in software. Keeping data immutable on the scale of whole operating systems and and even the whole word wide web is a necessary requirement for that.

Things that don't get referenced from anywhere anymore can be removed by a worldwide garbage collector.

  • RAM to HDD
  • application to application
  • computer to computer
  • file to file

An approach to implementation is "unison" (Link: [2]).

Software for 3D-modelling and beyond

(TODO: expand this chapter)

Atomistic modelling: From Tom's machine phase blog comes an example of the usage of Nanoengineer-1: DNA origami: from design to product

See: Data decompression chain

See: Visually augmented purely functional programming

Bad software design may undermine the "Disaster proof" property of globally used APM.

Further related information

Avoiding hidden state (a potential source of errors) can be compatible with interactive environments:

Relation of AP Technology to new computing paradigms

Related: Relations of APM to purely functional programming

What is likely to be a necessity

  • reversible computing

Deleting data produces heat proportional to the operation temperature. In super high density computing this needs to be avoided.

time-reversible computing low level programming language
High level programming language that match reversible computing best are functional programming languages like Liasp and Haskell.

What is absolutely not a necessity but could boost development speed

Widely known:

  • quantum computing (depends on reversible design)
  • neural network compution (deep learning)

Barely known:

Also maybe of interest:

  • ternary (mechanical) logic - higher radix economy but worse in transporting carries - (wikipedia)
  • digital usage of analog mechanical computing mechanisms (a few bit at a time) - link
  • usage of p-adic arithmetic in processors

Applications

Alternative computing architectures that boost certain types of problems can often be used for optimization:

  • circutry; layout;
  • optimal molecular topologies for bigger structural crystolecules e.g. brackets (aka Kaehler-brackets); ...

The various obstructing rifts in Software

  • ...

Related

External Links

Wikipedia:

Retrieved from "http://apm.bplaced.net/w/index.php?title=General_software_issues&oldid=16452"