What I would really like to see is more integration between terminal emulator and the shell (and commands you run in it). Currently in Fedora/GNOME the shell prompt is set up to make the terminal notify you (with a native GUI notification) about long running commands completing. That's cool, but we could go so much further.

For one, the shell could let the terminal know the current directory, hostname, username, git branch, current command, last command status, etc. etc.

The terminal then would display that info in its GUI, and perhaps even let you interact with it (imagine a GtkPopover for changing git branches and a Nautilus-like pathbar for the cwd!) We could then shrink back the prompt back to a simple dollar sign.

That would already be infinitely cool, but it gets better.

What if the shell could send its completions to the terminal and the terminal would display them in a native autocomplete widget (like in Builder)?

What if the terminal could track where the output of each command is and allow you to collapse/expand it with a GtkExpander? What if it could visually present each command and its output as this expandable card, along with the time it was issued and the time it took to run (like `time` outputs) and its exist code? And if it failed, paint that card red? And if it segfaulted, add a button to duck duck go the error, run ABRT or even run dnf debuginfo-install and coredumpctl gdb?

What if pagers like `less` could make use of the native scrolling instead of faking it by moving lines? What if `man` rendered the content to HTML (or GTK's native markup format) and presented it as an actual page, sort of like what you get by running `yelp man:foo` but right there in the terminal?

What if `ls` could make use of the native list/table widget and render file icons, just like Nautilus? What if you could expand the folders it outputs right there using mouse and get a tree view?

Don't tell me that fish or vim already display completions in a pop-up using some magical pseudographics. I'm not talking about abusing the old grid-of-characters model of terminals. I'm talking about making the shell and the terminal communicate better so we can use *native* (e.g. GTK+) widgets for what is traditionally done using text-based interfaces.


Mind, just like currently `ls` knows to columnize and colorize its output and `grep` knows to highlight the matches in red *when outputting to a terminal*, but fall back to plain text when piped to something, what I'm describing here would simply fall back to plain text when piped or run in a less magical terminal. So this is about enhancing the existing tools, not about breaking compatibility in any way.

And at the same time that would really get the terminal/shell experience from "emulating ancient hardware to be able to do stuff we haven't yet made a GUI for" to "a way to do advanced stuff with your computer using a command-based interface".

That would truly be a command-based interface for the 21th century. That would be something I'd be proud, not ashamed, to show off to Windows users as the advanced way to tune your system.

I've made a mock-up to illustrate my ideas about the next-gen terminal experience!

• the pathbar
• username, hostname and git branch displayed in the UI, shrinking the shell prompt back to just a $
• commands as cards
• syntax highlighting, including graying out the output a bit to differentiate it from commands themselves
• autocompletion (displayed in a native widget)
• built-in error handling options
• the time each command took (on the right)

All the widgets are interactive! You can click on the pathbar to `cd`, and the popovers on the bottom bar let you basically do `ssh`, `su` and `git checkout`; the git one additionally allows you to open the current repo in Gitg or in Builder.

Since everyone seemed to love my terminal mockups, I've made another one!

This is a further design exploration, showcasing:
• background jobs label/popover
• `ls` displaying an actual list of files! Of course, they are interactive, you can drag'n'drop them from here, click them to open, and right-click for more options.
• autosuggestions being contextual: here, they suggest you to re-run `apt` as root or read the docs
• a collapsed card
• `git` using real graphics

@bugaevc vaguely remember a project like this from a long time ago, an attempt to make the terminal show specific data more "intelligently"

oh found it: github.com/unconed/TermKit
(apparently also abandoned since 2011 !)

i guess the closest i otherwise can think of is the ipython-qt console which can show graphs &c inline

@brainblasted I wish! This is just a Glade .ui file plus a .png for the content.

Ask GNOME/Tilix devs/designers if you'd like to get a real-life version of this 😉

@bugaevc I was skeptical as you described it but I'd really like this if feasible

@bugaevc if you could have the bash-completion scripts indicate what sense each argument has (flag, file path, bare words), then the display could color / frame them, as well

@riking and not just that, the autocompletion window could display man snippets for options, like in these screenshots: github.com/railsware/upterm

@mbirth it sends commands to the shell (I imagine some escape sequence-based protocol, but I haven't thought about the specifics too much). See my other replies in this thread where I went into some more details about how this protocol should be shell- and toolkit-independent and how it somewhat resembles DWDs.

@bugaevc Ahh, so it's not a new shell, but some overlay which only works in a GUI? From the screenshots it looked like most features could be woven into a shell and thus be available on many systems without the need for a special terminal software.

@bugaevc it looks awesome, but wouldn't it add too much of unnecessary complexity to console software? Like, text-based stuff is so famously reliable exactly because it has very little moving parts, it's just spitting characters on screen, works every time, for everything. Text is very generalistic and universal, which is why it is the base for Unix.
And when you are trying to think about wrapping it into some GUI, you immediately lose in versatility - because GUI itself is a specialization, it always intentional, and it always has purpose. This purpose may always not align with what the user really wants. There's a lot of questions associated with the GUI, which you didn't have to answer whan you're making a CLI application, a lot of different decisions to make. That's why GUI design is its own damn field.

@drequivalent it would indeed add complexity, but I wouldn't call it unnecessary, I'd call it "better integration". Plus, most modern CLI programs/utils don't use plain text, they use libreadline, ncurses, terminfo, escape sequences, etc. etc., which is unreliable and to me sounds like it amounts to way more hacks than what I'm proposing.

@bugaevc @drequivalent I'm not sure how standard they are but there are various means for setting title, current directory (vte), and there are things like sixel which is a little what you want... See also the hyperlink support (not just url pattern matching like is the norm) which is are also ways to enrich terminal experience. Not sure how I feel about plumbing more and more over escape codes haha. #DBUSOVERXTERM

@bugaevc @drequivalent and there's terminology but I forget how that's done: enlightenment.org/about-termin.

Anyway, I agree there's room for improvement in the overall UI/workflow -- maybe time to borrow things from the likes of pharo? :D

@bugaevc @drequivalent also if we could just never terminfo/termcap at all that would be swell. And long overdue.

@drequivalent @bugaevc Yea that's what I was thinking. You'd have to create console programs that specifically had multiple I/O streams, some of them even object streams. It seems similar to what Microsoft was doing with Powershell, attempting to toss around full .NET objects with parameters instead of parsing/piping text IO.

The biggest problem would just be the tooling.

@drequivalent @bugaevc It pains me to see a response like this. It speaks from a platform of sheer ignorance.

First, there is no such field as "GUI design." It's just UI design; no C, no G, no T, nothing else. The principles of user interface design and layout transcends the mechanism of delivery.

@drequivalent @bugaevc Second, there are at least two different platform vendors who have invested heavily in usability testing in text-only displays: Digital and IBM.

The VAX VMS operating system has invested heavily in standardizing and regularizing their command-line interface in an effort to make it easier to use. They still have a vastly superior help system than either "man pages" or GNU's TexInfo mechanisms.

@drequivalent @bugaevc
In IBM's case, you might have heard of Common User Access standard, which influenced both Windows and OS/2 GUIs. Guess what? That whole standard evolved from prior research that went into making CICS panels on IBM mainframes easier and more productive to use.

So, yeah, 99% of those control key and function key presses you use to get around in your Windows desktop? And to a lesser extent, your Linux or BSD interface of choice?

Those. Came. From. IBM. MAINFRAMES.

@drequivalent @bugaevc Going back to Sergey's original proposition, this has **ALL** been done before. I encourage you to look up Symbolics Genera, a Lisp-based operating system for their line of Lisp computers. It is widely regarded as being the most productive, powerful, and approachable *command-line* interface ever seen on a computing platform. Yes, it requires a graphics/bitmapped display to run, but it's still first and foremost a *command-line.*

@drequivalent @bugaevc As a long-time user of a wide variety of command-line and graphical user interfaces, having a seamless integration between the two is absolutely something I'd be interested in. Especially if the great ideas of Genera can be excavated from the Symbolics graveyard and made more available on a contemporary platform.

Thank you for coming to my TED talk. (Or whatever it is kids these days say.)

@bugaevc have you tired any of the electron terminal applications that offer some of this functionality? If so, did you end up sticking with one?

@david uh, no. Have you? As far as I've seen, they're all in a prototype stage *and* abandoned.

@david Upterm (née Black Screen) and TermKit (technically web-based but not Electron-based) are the ones I'm talking about, they do offer some of these advanced features.

Hyper & Terminus don't seem to, and I don't see any reason to use them over a native terminal emulator app.

@bugaevc Fish actually can do a bunch of these things! It also has fancy things like commandline parameter auto-completion and inline auto-suggestions


@bugaevc and there are themes for it to do things like displaying the current git branch (and status via colour)

@theoutrider read the "don't tell me..." toot above :D

I do use Powerline! In fact, this thread started out as me talking to @federicomena about how I would like to see things Powerline does integrated into the native UI.

I love Fish. It's not POSIX, but you can always type "bash". I'm starting to like fancy emulators like Hyper.js a bit more too.

@bugaevc Idle thought: why not make the command grey, leaving white for output? The user knows what they've typed, but doesn't know what the output will be.

@wilfredh I've thought about that too! On the other hand, like half of the commands you type are more important that what they output (if anything), and it wouldn't look as pretty.

But this is just a mockup and I'm just a dev wishing for a better terminal experience, I'd rather let actual designers figure out what looks/works best.

@bugaevc This is great ! Even just being able to have something similar to [devicons](https://github.com/vorillaz/devicons) in `ls` results would be killer.
But I don't think it's the terminal's work to implement any of this.

@bugaevc this seems overcomplicated, I'd much rather have something like Acme or Plan 9's rio shells and build these things on top of that

@bugaevc it has a coolness factor to it but I would sleep better if I knew that whoever implemented this took a look at acme and rio first

@grainloom @bugaevc And, going back even further, Symbolics Genera and other Lisp Machine implementations.

@grainloom @bugaevc Both of you are not contradicting each other there :)

@ckeen @bugaevc I mean, it looks cool but uuugh, I hope it doesn't end up as its own OS and will integrate with the rest of the system

otherwise this just looks like ZSH with some GTK/Qt added

@grainloom @bugaevc

TBH all these fancy shell prompts and such are getting pretty ridiculous. If you are using a graphical UI it should take advantage of that. So far all we have are some weird terminal emulators with a baud rate setting.

That has to change

@ckeen @bugaevc @grainloom

I think you are undervaluing the efficiency of text.

Compare the logographic below to its (possible) translation:

"If a man utters this spell while pure, it assures going forth by day after his death" (see quora.com/How-does-Egyptian-hi for further info and translations).

We have a spacial problem (pixel to meaning ratio), a problem of composability and a problem of interpretation.

IMHO, text centric UI (like #Plan9's one) are simply too ahead of times.

@ckeen @bugaevc idk, I like a lot of the ideas in theory but if it's not as portable and modular as what we currently have, I don't see myself using it anytime soon
i'm not against making UIs better but I'm not sure how I feel about these ideas

how does it compare to what came before? how does it compare to Emacs?

@grainloom @bugaevc

None of this is portable in any way. That's why we are still stuck with what we have. Every wanted to know how the graphical shells change their title? There are some escape sequences for that. Icky stuff...

@ckeen @grainloom well, the idea *is* to make it portable. The shell and the terminal emulator would negotiate over some standard escape sequence-based protocol, and that protocol would not be bash- or GTK- specific, it'd work for any toolkit and shell combination. A bit like DWDs if you remember those.


I boosted because I had the exact same reaction of @grainloom : this interface is cool but cannot be done with a shell.

Some aspects of what @bugaevc propose are part of what I'd like to try in the #Jehanne UI (admitely a low priority task), but in a way that do not increase the complexity of the shell too much:

- the pathbar
- username
- folder specific tools (eg git/hg branch displayed in the UI)
- autocompletion

but I'd like to retain:
- editable output
- plumber integration

@ckeen @grainloom @bugaevc

Note that all of this could be built in Jehanne without modifying the shell, but we need a new screen multiplexer (aka rio) that somewhat integrate a text editor (eg sam) and a modified graphical terminal (aka win) to do the integration/framing.

This could hurt network transparency over 9P, though (I'm trying to design my file protocol so that it can preserve such transparency, but it's a loooong process, don't hold your breath).

Sign in to participate in the conversation
Mastodon for Tech Folks

This Mastodon instance is for people interested in technology. Discussions aren't limited to technology, because tech folks shouldn't be limited to technology either!